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1.  Summary 


This  final  technical  report  covers  the  two-year  duration  of  this  project,  from  April  16,  2015 
through  April  15,  2017. 

As  Air  Force  warfighting  missions  incorporate  increased  distributed  autonomy,  emergent 
global  behavior  may  arise  from  interactions  between  individual  autonomous  agents.  Example  Air 
Force  systems  that  may  in  the  coming  years  incorporate  increased  autonomy  resulting  in  little-to- 
no  direct  human  monitoring  and  intervention  include  drone  (UAV)  swarms  [1]  and  satellite 
constellations  [2].  Novel  methods  are  needed  to  ensure  such  distributed  cyber-physical  systems 
(DCPS)  have  trusted  assurance  to  meet  their  mission  requirements  and  only  their  mission 
requirements  in  spite  of  potential  emergent  distributed  behavior,  attacks,  and  failures. 
Understanding  distributed  emergence  and  being  able  to  respond  to  it  through  trusted  and  assured 
responses  will  allow  warfighters  to  continue  fighting  and  adapting  through  engagements,  enabling 
strategic  agility  in  Air  Force  missions.  Ultimately,  developing  theoretical  and  practical  tools  for 
understanding  and  responding  to  the  fundamental  phenomena  of  emergence  will  enable  the  Air 
Force  goal  to  fly,  fight,  and  win  ...  in  air,  space,  and  cyberspace. 

This  project  suggested  and  developed  the  use  of  scalable  formal  methods  in  mission  (1) 
specification  and  verification,  (2)  runtime  monitoring,  and  (3)  trusted  and  assured  control,  all 
conducted  in  conjunction  with  (4)  a  rigorous  evaluation  on  DCPS  with  prototypical  features  of 
modem  Air  Force  systems  such  as  UAV  swarms  and  satellite  constellations.  The  primary  research 
objectives  undertaken  were  to: 

•  Objective  1:  Develop  scalable  automated  formal  verification  methods  for  specifying  and 
verifying  trusted  global  DCPS  mission  behaviors  along  with  distributed  emergent 
behavior,  alleviating  state-space  explosion  by  exploiting  symmetries. 

•  Objective  2:  Develop  scalable  runtime  verification  and  monitoring  methods  relying  on 
both  formal  tools  and  heuristic  systems  to  detect  emergent  behaviors  and  violations  of 
global  mission  specifications  during  mission  operation. 

•  Objective  3:  Develop  runtime  assurance  (RTA)-like  trusted  control  methods  for  these 
distributed  systems  building  upon  the  foundational  theory  of  self- stabilization  of 
distributed  systems  [3-6]  and  the  Simplex  architecture  [7]  to  ensure  mission  specifications 
are  maintained  in  spite  of  emergent  distributed  behaviors  at  execution  time. 

•  Objective  4:  Evaluate  the  formal  specification  and  verification,  runtime  monitoring,  and 
control  methods  developed  in  the  other  objectives  on  challenging  DCPS  case  studies  with 
Air  Force  relevance,  particularly  swarm  robot  systems. 


2.  Introduction 

2.1.  Motivation 

Physical  systems  are  becoming  increasingly  dependent  upon  computers  and  software,  such 
as  in  emerging  embedded  and  cyber-physical  systems  (CPS),  where  networked  software  interacts 
with  physical  processes.  For  instance,  typical  modern  cars  utilize  dozens-to-hundreds  of 
microprocessors,  many  communications  buses,  and  a  complex  interconnection  between  sensors, 
actuators,  and  processors  [8-10].  In  the  design  and  development  process  for  most  engineered 
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systems  today  (including  CPS),  the  vast  majority  of  resources  are  devoted  to  ensuring  systems 
meet  their  specifications  [1 1,  12].  In  spite  of  significant  technical  advances  for  design  verification 
and  validation — such  as  model  checking,  hardware-in-the-loop  testing,  automatic  test  case 
generation  for  software,  and  sophisticated  simulators — there  are  frequent  safety  recalls  across  CPS 
industries  due  to  problems  between  cyber  and  physical  subcomponents.  For  example,  the 
Consumer  Product  Safety  Commission  (CPSC)  has  recalled  between  2010-2012  fire  alarm  and 
control  systems  from  Bosch,  Tyco-Grinnel,  and  Honeywell  for  failure  to  sound  alarms  and/or 
notify  fire  departments  [13-15],  the  Food  and  Drug  Administration  (FDA)  has  reported  the  leading 
cause  of  recent  medical  device  recalls  are  cyber-related  (tied  with  manufacturing  defects)  [16,  17], 
and  the  National  Highway  Traffic  Safety  Administration  (NHTSA)  has  recalled  hundreds  of 
thousands  of  2004-2005  and  2010-2014  Toyota  Priuses  due  to  drivetrain  software  problems 
causing  unexpected  stalls  [18,  19]  and  millions  of  2005-2010  Hondas  due  to  electronic  control 
model  software  causing  transmission  damage  [20].  Given  that  such  recalls  are  due  to  increased 
risk  of  physical  safety  (and  not  yet,  e.g.,  for  privacy  issues),  all  such  problems  are  inherently  cyber¬ 
physical.  As  future  networked  systems  like  robot  swarms,  the  smart  grid,  satellite  constellations, 
and  the  intelligent  transportation  system  increasingly  couple  distributed  agents  together,  emergent 
behavior  will  be  seen  to  spontaneously  arise.  Demonstrated  areas  of  distributed  emergence  through 
local  interaction  in  natural  and  engineered  systems  include  fish  schools  [21],  herds  [22],  highways 
[23,  24],  swarm  robotics  [25-27],  and  distributed  computing  [4,  28,  29]. 

2.2.  Air  Force  and  Department  of  Defense  Relevance 

Air  Force  Relevance  for  Strategic  Agility:  In  the  July  2014  report,  “America’s  Air  Force:  A  Call 
to  the  Future,”  Secretary  of  the  Air  Force  Deborah  Lee  James  outlines  a  three  decades  long 
strategic  plan  for  the  Air  Force,  centered  around  the  theme  of  strategic  agility.  In  this  strategy,  two 
technical  areas  of  relevance  to  this  project  are  highlighted,  namely  autonomous  systems  and 
unmanned  systems.  From  a  technical  standpoint  for  unmanned  systems,  strategic  agility  will 
enable  systems  with  little  human  supervision  to  “swarm,  suppress,  deceive,  or  destroy.”  For 
autonomous  systems,  strategic  agility  will  enable  moving  from  today’s  systems  that  are  “able  to 
execute  a  set  of  pre-programmed  functions”  to  tomorrow’s  systems  that  “will  be  better  able  to 
react  to  their  environment  and  perform  more  situational-dependent  tasks  as  well  as  synchronized 
and  integrated  with  other  autonomous  systems.”  The  work  completed  through  this  project  brings 
a  formal  perspective  to  what  it  means  for  systems  to  have  emergent  behavior,  such  as  what  may 
arise  in  the  challenging  environments  of  the  battlefield. 

Department  of  Defense  Relevance:  In  the  January  2013  report  “Resilient  Military  Systems  and 
the  Advanced  Cyber  Threat”  from  the  Defense  Science  Board,  a  number  of  broad  cyber  challenges 
and  opportunities  are  outlined  in  current  and  future  defense  systems.  Specific  recommendations  of 
the  task  force  report  include  “use  of  emerging  technology  developments  for  system  resilience,  such 
as  trust  anchors,  minimal  functionality  components,  simplified  operating  systems,  developing  a 
means  to  verify  compromise  of  fielded  systems  contributing  to  critical  missions,  creating  trust  in 
systems  built  with  un-trusted  components,  and  restoring  to  a  known  state.” 

Many  Air  Force  and  DoD  projects  are  currently  underway  related  to  these  areas.  For 
example,  the  High- Assurance  Cyber  Military  Systems  (HACMS)  projects  aims  in  part  to  develop 
verified  components,  the  BEDROCK  project,  high-assurance  microprocessors  are  in  development, 
techniques  for  determining  computer  component  intrusion  and  counterfeiting,  etc.  Verified 
operating  systems  like  seL4  and  verified  optimizing  compilers  like  CompCert  are  new  seminal 
results  toward  this  goal.  However,  the  work  completed  in  this  project  is  uniquely  differentiated 
from  all  existing  work  in  several  ways.  First,  this  work  focuses  on  emergent  behaviors  and 
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properties  that  may  arise  in  distributed  systems,  while  most  if  not  all  of  these  other  projects  and 
existing  approaches  focus  on  non-distributed  systems.  We  are  not  yet  to  the  state  where  distributed 
systems  may  be  fully  verified  (e.g.,  at  every  layer  of  the  OSI  network  model),  although  progress 
is  being  made  as  outlined  above  (and  in  e.g.,  verification  of  cryptographic  protocols,  key 
exchanges,  etc.).  Second,  the  work  of  this  project  is  not  a  clean-slate  approach  like  HACMS  and 
BEDROCK.  The  results  of  this  project  may  operate  within  the  constraints  defined  by  existing 
development  environments  practices,  and  the  reality  that  for  a  variety  of  reasons  (e.g.,  budgetary), 
there  is  additional  use  of  commercial  off-the  shelf  components  (COTS)  in  military  systems.  To 
operate  within  these  constraints,  we  investigated  both  fully  formal  approaches  and  semi-formal 
approaches  augmented  with  heuristic  approaches. 

2.3.  High-Level  Technical  Summary 

The  underlying  formal,  mathematical  framework  used  in  this  project  is  that  of  hybrid 
automata  [30],  which  are  finite-state  machines  augmented  with  real-valued  variables  that  evolve 
continuously  over  intervals  of  real  time.  Asynchronous  networks  composed  of  hybrid  automata 
[31]  are  useful  for  modeling  distributed  systems  that  interact  with  the  physical  world,  such  as  robot 
swarms  [5,  6],  air  traffic  control  systems  [32,  33],  autonomous  satellites  and  constellations  [2],  and 
distributed  electrical  microgrids  [34].  Desired  emergent  behaviors  include  phenomena  like 
flocking,  while  undesired  emergent  behavior  may  lead  to  catastrophic  mission  failure. 

Objective  1:  Scaling  Formal  Specification  and  Verification  for  Emergence  in  DCPS 

Objective  1  developed  design-time  formal  specification  and  verification  methods  for 
emergence  in  DCPS  modeled  as  networks  of  hybrid  automata  with  linear  and  nonlinear  dynamics. 
For  specifying  emergence,  new  specification  languages  for  CPS  using  hyperproperties  were 
developed  allowing  specification  of  frequency-domain  behavior  and  real-time,  real-valued 
behaviors  through  hyperproperties  for  signal  temporal  logic  (HyperSTL)  [35-37].  The  Passel 
verification  tool  [31,  38],  in  conjunction  with  a  small  model  theorem  [39],  an  invariant  synthesis 
procedure  [32],  and  a  symmetry-reduction  reachability  method  [40],  enabled  the  first  fully 
automatic  verification  of  safety  (aircraft  separation)  for  the  Small  Aircraft  Transportation  System 
(SATS)  landing  protocol  (a  part  of  the  NASA/FAA  NextGen  program  [41-48]).  Through  this 
objective,  we  built  upon  these  approaches  for  addressing  the  state-space  explosion  problem  to  scale 
verification  methods  to  larger  DCPS  than  previously  possible,  as  well  as  developed  a  new 
verification  tool,  HyST  [49]. 

Objective  2:  Detecting  Emergence  at  Runtime:  Specification-Based  Runtime  Monitoring 

Objective  2  developed  formal  and  heuristic  runtime  monitoring  verification  methods  for 
emergence  in  DCPS,  using  both  model-based  and  model-free  approaches.  Model-based  methods 
rely  on  formal  methods  tools  and  inherently  are  subject  to  scalability  problems,  while  model-free 
approaches  are  heuristic,  as  they  are  both  unsound  and  incomplete,  but  scale  better.  Together,  the 
methods  rely  on  monitoring  asynchronous  distributed  and  hybrid  systems  at  runtime  and  in  real¬ 
time,  and  build  upon  both  model-free  and  model-based  approaches  developed  by  our  group  [1,7, 
50].  For  DCPS,  we  extended  an  invariant  inference  tool  called  Hynger  (HYbrid  iNvariant 
GEneratoR)  [50]  that  instruments  arbitrary  MathWorks  Simulink/Stateflow  (SFSF)  models  to 
generate  candidate  invariants  over  input  and  output  variables  [51]. 

Objective  3:  Assured  Control  in  Spite  of  Emergence  with  Real-Time  Reachability  and  Self- 
Stabilization  for  Distributed  Simplex 

Objective  3  developed  control  methods  to  ensure  desirable  or  avoid  undesirable  emergent 
distributed  behavior  at  runtime,  by  leveraging  the  Simplex-based  RTA  framework  using  real-time 
reachability  of  networks  of  hybrid  automata  in  conjunction  with  self- stabilization  [3-6,  52,  53]  of 
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the  distributed  system.  Monitoring  predicates  over 
physical  variables  and  their  continuous  evolution  over 
time  was  performed  with  real-time  hybrid  systems 
reachability  and  runtime  monitoring  of  emergent 
behavior  specified  and  detected  in  Objectives  1  and  2. 
Objective  4:  Evaluating  Analysis,  Monitoring,  and 
Control  of  Emergence  in  DCPS 
Objective  4  is  evaluating  the  novel  methods  for 
distributed  emergence  developed  in  the  previous 
objectives.  We  performed  analytical  analysis, 
simulations,  laboratory  experiments,  and 
demonstrations  using  a  swarm  of  autonomous  agents, 
particularly  commercially  available  quadrotor  drones. 
Typical  safety  properties  that  arise  are  collision 
avoidance  and  convergence  to  some  desired 
configuration  and/or  location  [52],  and  emergent 
properties  may  be  consensus,  flocking,  or  unwanted 


Figure  1:  High-level  overview  of  the  DCPS 
modeling  framework,  where  each  agent 
(participant)  in  the  distributed  system  is  modeled 
as  a  hybrid  automaton,  and  a  network  is  composed 
of  these  automata  that  may  communicate  through 
a  potentially  lossy  and  adversarial  channel. 


oscillatory  movements  due  to  failures,  attacks,  communication  delays,  etc.  Our  results  in  similar 
studies  include  verification  of  autonomous  satellite  maneuvers  [2],  flocking  in  swarm  robotics  in 
spite  of  failures  [5,  6],  and  planar  robotics  [52], 


3.  Methods,  Assumptions,  and  Procedures 

3.1.  Overview 

The  objectives  summarized  in  the  previous  section  were  undertaken  through  the  following 
technical  procedures  and  methods. 

3.2.  Technical  Procedures 

3.2.1.  Scaling  Formal  Specification  and  Verification  for  Emergence  in  DCPS 

We  first  developed  design-time  formal  specification  and  verification  methods  for 
emergence  in  DCPS  modeled  as  networks  of  hybrid  automata  with  linear  and  nonlinear  dynamics. 
We  developed  a  verification  framework  for  modeling  DCPS  as  networks  of  hybrid  automata  that 
interact  through  discrete  transitions  [31,  32,  39,  40],  The  Passel  verification  tool  [31,  38],  in 
conjunction  with  a  small  model  theorem  [39]  and  an  invariant  synthesis  procedure  [32],  enabled 
the  first  fully  automatic  verification  of  safety  (aircraft  separation)  for  the  Small  Aircraft 
Transportation  System  (SATS)  landing  protocol  (a  part  of  the  NASA/FAA  NextGen  program  [41- 
48]).  Extending  Passel  and  its  theoretical  framework  to  emergence  properties  for  swarm  robotics 
first  requires  developing  formal  definitions  and  specifications  of  emergence  properties  in  DCPS. 

3.2. 1.1.  Formally  Defining  and  Specifying  Emergent  Behavior  in  DCPS 

The  approach  is  to  specify  emergence  as  sets  of  invariant  properties  over  the  local  states  of 
individual  automata,  to  describe  the  global  behavior  of  the  entire  distributed  systems.  For  example, 
invariants  allow  specifying  either  creation  or  absence  of  emergence  of  consensus  or  flocking 
behavior.  DCPS  are  naturally  parameterized  by  a  number  of  interacting  agents,  for  instance,  the 
number  of  robots  in  a  swarm.  We  define  absence  and  presence  of  emergence  properties  using 
invariants,  integrate  the  formal  specification  of  emergence  into  an  extension  of  the  Passel 
verification  tool  [31,  38]  with  a  new  software  tool  HyST  [49],  and  evaluate  synthesizing 
implementations  in  a  correct-by-construction  manner  [54], 
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Figure  2:  Example  of  emergent  flocking  behavior  in  three  dimensions  with  a  system  of  N  =  64  agents.  The  left  frame 
is  at  an  initial  condition  and  the  right  frame  illustrates  the  flocking  formation  after  36  seconds  of  runtime.  The  agent 
positions  are  denoted  by  green  circles,  their  velocities  by  green  vectors,  and  a  red  vector  indicates  their  desired 
heading.  Blue  lines  between  agents  are  drawn  if  their  distances  are  approximately  spaced  by  some  desired  flocking 
spacing  ry. 

Figure  2  shows  emergent  flocking  behavior  in  a  simulation  of  the  Olfati-Saber  algorithms 
[25-27]  that  rely  only  on  local  communication  between  the  agents.  Flocking  is  not  specified 
anywhere  in  the  system  description,  instead,  it  emerges  dynamically  as  a  property  of  the  system 
over  its  execution.  One  definition  of  flocking  is  that  all  agents  are  spaced  equally  from  all  their 
neighbors,  which  may  be  specified  mathematically  as: 

Vi  G  [N],Vj  G  Q:  ||xj  —Xj ||  =  ry,  (1) 

where  i,j  come  from  a  set  of  agent  identifiers  [N]  =  {1, ... ,  /V) ,  xt ,Xj  are  real  vectors  of  an 
appropriate  dimensionality  (e.g.,  3  for  the  example  of  Figure  2),  ||  •  ||  is  an  appropriate  norm  (e.g., 
say  the  2-norm),  C*  is  a  set  of  communication  neighbors  of  i  (e.g.,  Cj  =  {j  G  [N]i  ||xj  —  Xj  ||  <  rc } 
for  some  communication  radius  rc),  and  ry  >  0  is  some  desired  flocking  spacing  [1,  5,  6],  Note 
that  non-ideal  spacing  may  easily  be  incorporated,  e.g.,  to  define  a  flock  as  states  where  agents  are 
approximately  spaced  by  ry,  such  as  ry  +  €f  for  some  small  6f.  Control  algorithms  to  enable  such 
emergent  behavior  do  not  a  prior  specify  anything  about  the  behavior,  rather  it  arises 
spontaneously.  Other  emergent  properties  of  interest  for  such  systems  include  collision  avoidance, 
which  may  be  specified  in  a  similar  format,  such  as: 

Vi,j  E[N]:\\xi- Xj\\>rs,  (2) 

where  all  quantities  are  as  before  and  rs  <  ry  is  a  desired  spacing  amount.  Note  that  these  emergent 
behaviors  are  potentially  in  conflict  with  one  another:  flocking  mandates  agents  come  sufficiently 
close  together,  while  safety  mandates  agents  do  not  come  too  close  together. 

From  a  specification  standpoint  of  these  two  different  forms  of  emergent  behavior 
described  in  (1)  and  (2),  there  are  several  similarities.  First,  the  class  of  formulas  these 
specifications  come  from  is  quite  similar.  These  are  both  specified  using  universal  quantification 
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Figure  4:  Alternative  specifications  of  flocking  emergence  exist  like  bird  vees.  This  is  a  planar  scenario  created  by 
adjoining  two  one -dimensional  flocks  (platoons)  about  an  appropriate  angle.  The  middle  figure  shows  a  phase  space 
plot  of  the  trajectories  of  all  agents,  and  the  right  figure  shows  the  planar  coordinates  of  all  agents  as  they  evolve  over 
time  while  moving  and  rotating  in  the  plane.  By  composing  formally  verified  primitives  (the  exponentially  stable  one¬ 
dimensional  flocking  algorithm),  sophisticated  and  verified  planar  formation  control  is  achieved.  High-level  mission 
specifications  and  flock  formation  parameters  (such  as  the  angles,  where  to  move,  etc.)  may  be  specified  in  a  temporal 

followed  by  a  quantifier- 
free  formula  over  reals. 
Both  of  these 
specifications  of 

emergent  behavior 
almost  fall  into  the 
restricted  class  of  first- 
order  logic  (FOL) 
supported  in  the 
theoretical  framework 
developed  for  uniform 
verification  of  safety 
properties  in  networks  of 
hybrid  automata,  with 
automated  reasoning 
methods  implemented  in  the  Passel  software  tool  [31,  32,  39].  To  highlight  one  subtlety,  note  that 
an  alternative  way  to  represent  the  set  of  communication  neighbors  of  an  agent  is  using  a  set¬ 
valued  variable,  i.e.,  an  array. 


logic  like  linear  temporal  logic  (LTL). 

x 104  540 

2.5 


x  104 


Figure  3:  The  left  image  shows  divergent  emergent  behavior  when  trying  to  use 
distributed  flocking  control  algorithms  with  realistic  system  constraints,  particularly 
(1)  actuator  saturation,  (2)  asynchrony,  and  (3)  communication  delays,  and  the  right 
image  shows  partial  emergence  of  flocking  for  these  factors. 


However,  extensions  are  needed  to  support  planar  and  three-dimensional  specifications  of 
flocking,  extensions  to  the  restricted  class  of  FOL  supported  by  the  small  model  theorem  [39] 
exploited  by  Passel.  Specifically,  the  specification  of  the  two-norm  is  a  polynomial  expression 
over  the  reals,  while  Passel  has  only  been  used  so  far  on  linear 
expressions.  Thus,  a  first  objective  is  to  extend  Passel  to  support 
polynomial  expressions.  Next,  realistic  systems  have  continuous 
dynamics  specified  by  linear  or  nonlinear  ordinary  differential 
equations  (ODEs),  while  the  modeling  language  supported  by  Passel 
does  not  currently  allow  this.  Additionally,  an  extension  of  the  small 
model  theorem  for  these  scenarios  is  required,  as  it  also  only  allows 
linear  expressions,  while  the  solutions  of  ODEs  may  generally 
involve  special  functions  and  transcendentals.  This  extension  to  the  pigure  5.  pmergent  flocking 
theoretical  basis  of  Passel  was  made  and  integrated  within  HyST  with  four  groups  of  agents  usina 

platooning  algorithms. 
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Figure  6:  Emergent  planar  flocking  behavior  under  ideal  conditions 
required  by  existing  distributed  control  algorithms  without  attacks, 
failures,  control/actuation  saturation,  asynchrony,  or  communication 
failures. 


[49].  This  extension  is  feasible, 
under  the  assumption  that  the 
continuous  dynamics  of  an  agent  i 
do  not  directly  depend  upon  those 
of  an  another  agent  j .  That  is  to 
say,  their  continuous  interactions 
are  decoupled.  They  may  however 
interact  discretely,  through  for 
instance  communication  or 
computer-sampled  sensing.  This 
together  leads  to  the  next 
approach,  of  extending  the 


modeling  language  and  theoretical  basis  of  Passel  to  support  both  decoupled  linear  and  nonlinear 
ODEs. 


3. 2. 1.2.  Formal  Verification  using  State-Space  Reductions  in  Hybrid  Automata  Networks 

Previous  limitations  of  verification  methods  for  DCPS  required  each  automaton  in  the 
network  to  have  rectangular  dynamics  (x  G  [a,  b]  for  real  constants  a  <  b).  While  many  systems’ 
dynamics  may  be  reasonably  over- approximated  as  rectangular  differential  inclusions,  it  is  critical 
to  extend  the  framework  and  results  to  support  linear  and  nonlinear  differential  equations.  The 
Passel  verification  tool  and  its  theoretical  basis  was  extended  within  HyST  to  support  DCPS  with 
linear  and  nonlinear  continuous  dynamics,  enabling  it  to  realistically  specify  and  verify  swarm 
robotics  case  studies  with  emergence  by  exploiting  symmetry-reduction  methods  for  reachability 
[31,  40,  55-63]  and  small  model  theorems  [39]  for  proving  inductive  invariants  to  establish.  Since 
these  methods  are  sound  and  consider  all  system  behaviors  and  permutations,  they  have  the 
capability  to  establish  the  presence  or  absence  of  emergence  over  the  evolution  of  these  DCPS. 

The  main  technical  challenge  in  utilizing  such  methods  (for  any  formal  model)  is  the  state- 
space  explosion  problem  (referred  to  as  the  “curse  of  dimensionality”  in  other  fields)  [31,  55-57, 
64-68],  which  is  that  the  size  of  the  state-space  grows  exponentially  in  the  number  of  components 
(see  Figure  7).  For  example,  small  model  theorems  [31,  39,  67,  69-75]  allow  for  formally  verifying 
safety  and  liveness  properties  of  arbitrarily  large  parameterized  networks  of  communicating 
automata  using  finite  (and  typically  small)  equivalent  systems.  The  “small  model”  here  refers  to 
the  size  of  models  in  the  formal  logic  sense  that  are  necessary  to  consider  in  deductive  proofs.  That 
is,  a  model  is  a  satisfying  assignment  to  a  sentence,  and  the  size  refers  to  the  largest  size  of 
satisfying  assignments  that  need  be  considered,  and  not  to  the  size  of  the  system  model  itself, 
although  there  is  clearly  a  relationship  between  the  two. 

State-space  explosion  is  a  challenging  problem  in  verification,  and  in  [39]  we  developed  a 
small  model  method  for  verifying  safety  properties  of  arbitrarily  large  networks  of  hybrid  automata 
by  verifying  finite  networks.  For  example,  in  an  air  traffic  control  system,  each  aircraft  may  be 
modeled  as  a  hybrid  automaton  and  a  safety  specification  is  that  no  two  aircraft  ever  come  too 
close  to  one  another  to  establish  that  aircraft  never  collide.  A  major  focus  of  this  research 
community  is  to  develop  mathematical  and  software  tools  to  verify  that  CPS  design  models  meet 
their  requirements.  Of  course,  automation  is  challenging  for  a  variety  of  reasons,  such  as  the  state- 
space  explosion  problem  and  the  combinations  of  discrete  and  continuous  dynamics.  Significant 
effort  was  spent  developing  a  software  tool  called  Passel — a  collective  noun  meaning  a  large  group 
of  indeterminate  number — for  automatic  verification  of  parameterized  CPS,  and  all  methods  in 
this  project  were  implemented  algorithms  in  publicly  available  tools  (HyST,  Hynger,  StarF, 
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rtreach,  specified  in  the  deliverables).  Systems  are 
modeled  in  Passel  as  hybrid  automata,  and  the 
tool  generates  the  CPS  semantics  in  a  restricted 
subclass  of  first-order  logic  (FOL)  over  reals, 
bitvectors,  and  integers.  Passel  leverages  recent 
advances  in  satisfiability  modulo  theories  (SMT) 
solvers.  Passel  exploits  the  small  model  theorem 
we  developed  to  reduce  verification  for  networks 
composed  of  arbitrarily  many  (countably  infinite) 
hybrid  automata  to  checking  a  network  with  a 
(small)  finite  number  [39].  Abstraction  results 
like  this  enable  scalable  verification,  and  allow 
Passel  to  automatically  prove  inductive  invariants 
by  checking  validity  of  appropriate  FOL 
formulas.  Passel  has  been  applied  to  verify  CPS 
examples  like  the  Small  Aircraft  Transportation 


Automata,  N 

Figure  7:  Illustration  of  the  state-space  explosion 
problem  for  the  Small  Aircraft  Transportation  System 
(SATS)  case  study  modeled  as  networks  of  hybrid 
automata  [31,  39,  33],  and  using  a  small  model  theorem 
to  address  the  problem. 


10000.00 


System  (SATS)  landing  protocol  in  NASA/FAA  NextGen  program  [41-48]. 

Reductions  in  Formal  Verification :  Symmetry-reduction  methods  [31,  40,  55-63]  similarly  allow 
for  formally  verifying  systems  with  large  spaces  by  only  exploring  small  equivalences  classes  of 
the  large  state  space.  For  example,  in  preliminary  results  [31,  40]  shown  in  Figure  8  allow  for 
verification  of  significantly  larger  networks  of  hybrid  automata  than  existing  methods  (e.g.,  in 
PHAVer  [76]  or  SpaceEx  [77]).  In  preliminary  results  [31,  40]  consider  systems  that  have  on  the 
order  of  2 130  discrete  states  (growing  at  N (4N)n ,  see  Figure  7)  as  well  as  on  the  order  of  N  — 
20  to  hundreds  of  continuous  variables ,  where  N  is  the  number  of  automata  in  the  network  (the 
x-axis  in  Figure  7  and  Figure  8).  No  other  tool  can  support  such  large  state  spaces  with  a 
combination  of  both  complex  discrete  and  continuous  behaviors  (e.g.,  [76-79])  and  the  closest 
comparable  tool  is  Uppaal  [80]  (but  that  does  not  support  as  general  dynamics).  Leveraging  these 
results,  we  developed  the  first  formal  verification  of  emergent  properties  like  flocking  in  DCPS. 
3.2.2.  Detecting  Emergence  at 
Runtime:  Specification- 

Based  Runtime  Monitoring 
Objective  2  was  the 
development  of  runtime 
monitoring  verification 

methods  for  emergence  in 
DCPS,  using  both  model-based 
and  model-free  approaches. 

Model-based  methods  rely  on 
formal  methods  tools  and 
inherently  are  subject  to 
scalability  problems,  while 
model-free  approaches  are 
heuristic,  as  they  are  both 
unsound  and  incomplete,  but 
scale  better.  Together,  the 
methods  rely  on  monitoring 
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Figure  8:  Symmetry-reduced  reachability  of  hybrid  automata  networks 
implemented  in  the  Passel  verification  tool  [31,  40,  38],  which  addresses  the 
state-space  explosion  problem  and  allows  significantly  larger  problem  size 
than  existing  state-of-the-art  methodology  (in  PH  AVer). 
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asynchronous  distributed  and 
hybrid  systems  at  runtime  and 
in  real-time,  and  builds  upon 
both  model-free  and  model- 
based  approaches  developed 
by  our  group  [1,  7,  50].  While 
the  model-based  design 
framework  and  typical  formal 
verification  problem  assumes 
a  system  model  <A  with 
formal  semantics  is  available, 
this  is  rarely  the  case  in  the 
current  state  of  engineering 
practice.  We  developed  an 
invariant  synthesis  tool  called 
Hynger  (HYbrid  iNvariant 
GEneratoR)  [50]  that  instruments  arbitrary  Simulink/Stateflow  (SLSF)  block  diagrams  for  input  to 
the  Daikon  invariant  finder  [81,  82]  to  generate  candidate  invariants  over  the  input  and  output 
variables  of  every  block  in  a  diagram.  The  internals  of  the  SLSF  blocks  may  be  unknown,  be 
compiled  machine  code,  actual  systems,  etc.  Such  heuristic  methods  scale  better  than  formal 
methods  alone.  However,  if  the  internals  are  known  and  formal  models  are  available,  the 
candidates  may  then  be  checked  to  be  actual  invariants  using  tools  like  Passel  [31],  HyCreate  [83], 
SpaceEx  [77],  etc.,  so  these  heuristic  methods  enable  scalable  usage  of  formal  tools  for  monitoring 
invariants. 


Figure  9:  Hynger-based  formal  and  heuristic -based  invariant  inference  for 
emergent  behavior  in  DCPS.  Sets  of  candidate  invariants  are  generated  to  either 
monitor  the  sets  of  invariants  themselves  and  how  they  change  over  time,  as 
well  as  prove  that  these  candidates  are  actual  invariants  for  RTA. 


3.2.2. 1.  Model-Free  and  Model-Based  Invariant  Inference  and  Synthesis  for  Emergence  in  DCPS 

We  first  extend  the  invariant  inference  methodology  to  distributed  CPS  from  individual 
systems  currently  supported.  Combined  with  emergence  specified  as  invariants,  this  allows  for 
identifying  the  presence  or  absence  of  emergent  behavior  in  DCPS  at  runtime.  While  not  all 
interesting  specifications  of  emergent  behavior  may  be  found  as  invariants,  many  examples  can, 
such  as  those  for  flocking  and  collision  avoidance  in  (1)  and  (2). 

The  overall  methodology  is  depicted  in  Figure  9.  A  CPS  model  or  implementation  is 
provided  as  a  SLSF  diagram  Jl.  The  SLSF  diagram  is  instrumented,  then  the  SLSF  diagram  is 
executed  to  generate  a  set  of  sampled,  finite -precision  traces  T  for  each  initial  condition  9  in  a  set 
of  initial  conditions  0,  which  effectively  corresponds  to  a  test  suite.  The  traces  are  analyzed  using 
dynamic  analysis  methods,  such  as  Daikon,  to  generate  a  set  of  candidate  invariants  <t>.  each 
element  (p  of  which  may  be  checked  as  actual  invariants  if  c/Z  corresponds  to  a  formal  model  (e.g., 
a  hybrid  automaton),  then  a  model  checker  may  be  employed  to  see  if  it  is  an  actual  invariant  (p, 
and  the  set  of  actual  invariants  <t>  is  collected.  Next,  each  candidate  invariant  (p  £  <£>  is  projected 
(restricted)  onto  the  subset  of  physical  variables  to  yield  a  candidate  physical  invariant  (pP  and 
corresponding  set  d>P.  Now,  &P  corresponds  to  the  candidate,  inferred  physical  invariants  from 
the  perspective  of  the  DCPS.  The  candidate  sets  of  invariants  and  proved  invariants  are  used  for 
runtime  monitoring  and  verification  (RMV)  and  runtime  assurance  (RTA). 


To  formalize  the  problem,  an  extension  of  hybrid  input/output  automata  (HIOA)  was 
developed  [53,  84-86],  called  cyber-physical  input/output  automata  (CPIOA)  [35].  In  addition  to 
partitioning  variables  into  local,  input,  and  output  sets,  each  of  these  sets  of  variables  are  further 
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partitioned  into  cyber  and  physical  variables.  Then,  when  states 
(or  formulas  used  to  symbolically  represent  states)  are  restricted 
to  the  set  of  cyber  (respectively,  physical)  variables,  the 
specifications  then  correspond  to  the  cyber  (respectively, 
physical)  specification.  In  practical  software  implementations 
using  e.g.,  C  and  SLSF  models,  the  physical  variables  can  be 
specified  using  a  subtyping  of  the  usual  types  for  approximation 
of  reals  (e.g.,  a  physical  variable  is  a  subtype  of  double  floating¬ 
point  or  fixed-point  types).  Techniques  building  on  taint  analysis 
of  programs  are  used  to  identify  the  effects  of  all  physical 
variables  in  CPS  [87] . 

We  utilize  both  dynamic  and  static  analyses  of  CPS 


SpaceEx  model 

Model  analysis 

Verification 

(Converter) 


Simulink  model 


Simulation 
Code  generation 

Figure  10:  Correct-by-construction 
implementations  of  DCPS,  starting 
from  a  formal  model  (e.g.,  as  a 
SpaceEx  hybrid  automaton  network) 
using  a  sound  translation  to 
implementations  as  Simulink 
models. 


models  to  infer  the  cyber-physical  specifications  of  emergence.  When  models  with  formal 
semantics  (e.g.,  CPIOA)  are  available,  static  analysis  in  the  form  of  reachability  analysis  may  be 
employed  to  determine  invariant  specifications.  If  no  such  formal  models  (or  potentially  no  models 
and  even  only  black-box  implementations  are  available),  one  may  employ  dynamic  analysis  by 
executing  (or  simulating)  the  systems  under  consideration  to  generate  sets  of  executions  (or 
sampled  approximate  traces,  due  to  inherent  inaccuracies  of  simulation  on  finite -precision  digital 
computers).  We  developed  a  methodology  within  Hynger  for  instrumenting  arbitrary  SLSF 
diagrams  (that  may  potentially  have  known  or  unknown  models  or  system  implementations)  to 
generate  output  traces  in  the  format  compatible  with  the  Daikon  dynamic  invariant  inference  tool 
[81,  82].  The  SLSF  blocks  may  be  unknown  models  or  even  system  implementations  since  from 
the  point  of  view  of  SLSF,  the  only  information  required  for  blocks  are  variable  values  at  block 
inputs  and  outputs  and  when  that  information  is  updated.  For  instance,  SLSF  may  be  integrated 
with  hardware/software-in-the-loop  simulation,  and  for  these  purposes,  some  blocks  represent 
models  to  be  simulated  and  have  information  necessary  to  perform  simulation,  while  other  blocks 
actually  correspond  to  implementations  that  have  been  interfaced  to  provide  necessary  data  to 
SLSF.  Since  physical  variables  evolve  according  to  ODEs,  their  invariants  may  involve  nonlinear 
and  transcendental  functions.  Nonlinear  (polynomial)  invariants  [88],  disjunctive/max-plus 
invariants  [89,  90],  and  simulation-based  verification  (which  effectively  define  invariants  from 
dynamic  analysis)  [91]  may  be  used  to  greatly  expand  the  classes  of  invariants  that  may  be  found. 
If  formal  models  are  available,  one  may  check  if  the  inferred  invariants  are  actual  invariants  using 
hybrid  systems  model  checkers  such  as  SpaceEx  [77],  HyCreate  [83],  and  Passel  [31,  38],  Physical 
dynamics  and  specifications  thereof  are  formalized  in  a  mechanized  manner,  similar  to  the 
numerical  simulations  formalized  in  ACSL  [92]  for  Frama-C  [93]. 

Using  the  formalized  distributed  emergence  inference  methods,  offline  algorithms  to 
identify  emergence  were  developed.  As  detailed  in  Figure  9,  this  results  in  SMT  validity  and 
satisfiability  checks  over  formulas  symbolically  representing  the  candidate  invariants.  We 
implemented  specification  inference  methods  in  software  tools.  A  software  tool  is  developed 
implementing  the  algorithms  developed  in  the  other  objectives  to  solve  the  emergence  inference 
problem  at  design  time.  The  software  tool  is  called  Hynger  (for  Hybrid  iNvariant  GEneratoR)  and 
integrates  with  typical  CPS  development  environments  (Mathworks  Matlab/Simulink)  as  well  as 
formal  analysis  tools  for  hybrid  systems,  such  as  Passel  [31,  38]  and  SpaceEx  [77],  This  leverages 
extensive  experience  using  SMT  solvers  [94,  95]  such  as  Z3  [96]  used  by  the  Passel  tool  [31,  38, 
40]  for  the  satisfiability/validity  checks.  Case  study  models  (and  the  testbed  described  in  Objective 
4)  are  developed  to  evaluate  the  inference  methods. 


Approved  for  Public  Release;  Distribution  Unlimited. 

10 


We  investigated  richer  specification  languages,  such  as  temporal  logics  (e.g.,  linear 
temporal  logic  [LTL]  or  computation-tree  logic  [CTL],  etc.),  as  well  as  real-time  temporal  logics 
(e.g.,  metric  temporal  logic  [MTL],  metric  interval  temporal  logic  [MITL],  signal  temporal  logic 
[STL]  [97],  etc.  [98]).  With  richer  specification  languages  like  LTL,  richer  techniques  are  be 
necessary,  and  ideas  such  as  Angluin’s  learning  algorithm  [99,  100]  or  counterexample-guided 
synthesis  [97,  101]  to  infer  specifications  (i.e.,  finite-state  automata  for  LTL  and  parameters  for 
STL)  from  executions  were  investigated.  The  detection  of  emergence  becomes  more  complex,  as 
instead  of  satisfiability  checks  between  invariants  to  determine  inclusions,  language  inclusions 
must  be  checked.  To  work  with  C  code.  Daikon  must  utilize  appropriately  instrumented  binaries 
using  Valgrind  via  its  Kvasir/Fjalar  frontends  [82].  This  makes  it  difficult  to  use  on  non-x86/x86- 
64  platforms,  which  is  a  serious  limitation,  as  most  embedded  platforms  utilize  other  architectures 
(e.g.,  ARM,  AVR,  PIC,  8051,  MSP430,  etc.).  Due  in  part  to  these  limitations,  the  methodology 
instruments  architecture-independent  SLSF  diagrams  to  generate  traces  in  the  input  format 
compatible  with  dynamic  analysis  tools  like  Daikon.  The  Hynger  tool  takes  an  arbitrary  SLSF 
model,  instrument  it,  then  analyze  the  resulting  traces  with  dynamic  analysis  to  identify  broad 
classes  of  emergent  behavior. 

3. 2.2. 2.  Runtime  Assurance  and  Runtime  Verification  for  Emergence  in  DCPS 

Next,  the  candidate  invariants  detected  using  the  Hynger  and  Daikon  tools  may  be 
monitored  at  runtime  to  enable  a  mntime  assurance  framework  like  the  ClearView  system  for 
distributed  (purely  software)  systems  [102,  103].  While  technically  unsound  and  incomplete, 
practically,  given  a  sufficiently  large  test  database,  the  candidate  invariants  correspond  well  to  the 
expected  behaviors  of  the  system,  and  serve  as  abstractions  of  all  internal  behavior.  At  runtime 
when  analyzing  traces  over  finite  times,  if  the  candidate  invariants  inferred  are  not  implied  by 
known  candidates  then  a  suspicious  scenario  is  flagged  (such  as  an  attack  [102,  103],  emergent 
behavior,  etc.).  We  investigated  and  use  distributed  global  predicate  and  state  detection  algorithms 
that  rely  on  minimal  communication,  building  upon  seminal  results  of  Chandy,  Misra,  and 
Lamport  [104-106].  Self-stabilization  [4]  is  used  as  a  tool  to  formalize  the  emergence  specifications 
and  their  evolution  over  time  in  the  distributed  systems  (as  invariants,  i.e.,  predicates  of  state 
space). 

3.2.3.  Assured  Control  in  Spite  of  Emergence  with  Real-Time  Reachability  and  Self- 
Stabilization  for  Distributed  Simplex 

Objective  3  is  the  development  of  control  methods  to  ensure  desirable  or  avoid  undesirable 
emergent  distributed  behavior  at  mntime,  by  leveraging  the  Simplex-based  RTA  framework  using 
real-time  reachability  of  networks  of  hybrid  automata  in  conjunction  with  self-stabilization  [3-6, 
52,  53]  of  the  distributed  system.  Monitoring  predicates  over  physical  variables  and  their 
continuous  evolution  over  time  is  performed  with  real-time  hybrid  systems  reachability.  We 
developed  a  methodology  for  runtime  assurance  in  Simplex- architecture  RTA  systems  using  real¬ 
time  reachability  for  a  single  hybrid  automaton  [7].  These  results  are  restricted  to  a  single  hybrid 
automaton,  and  require  extensions  to  DCPS.  Since  modern  DCPS  are  complex,  it  may  be  infeasible 
to  determine  all  specifications  and  possible  emergence  between  all  subcomponents  at  design  time. 
We  developed  online  runtime  monitoring  and  verification  methods  for  the  inferred  candidate 
specifications  of  emergence,  and  combine  these  monitoring  methods  with  real-time  algorithms  for 
detecting  emergence  at  runtime.  When  emergence  is  identified  at  runtime,  a  runtime  assurance 
framework  building  on  supervisory  control  ensures  safe  DCPS  runtime  operation  in  spite  of 
emergence. 
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For  some  of  the  analysis,  we  assume  formal  hybrid 
automata  models  are  available,  which  may  not  be  the 
case  for  practical  CPS  that  are  designed  using  more 
typical  industrial  tools  such  as  Mathworks 
Simulink/Stateflow  ( SLSF ).  To  alleviate  this  issue,  we 
investigated  a  new  design  paradigm,  where  the  plant, 
controller,  and  their  interfaces  are  designed  formally  as 
hybrid  automata,  then  are  translated  to 
implementations  as  SLSF  diagrams  (see  Figure  10) 
[54].  This  paradigm  of  designing  with  formal  models, 
then  instantiating  implementations  is  attractive,  as  both 
simulation  and  verification  may  be  conducted  with  the 
formal  model,  then  an  implementation  may  be  derived 
that  is  guaranteed  to  have  the  same  behaviors.  The 
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Figure  1 1 :  Overview  of  the  Simplex  architecture 
where  an  unverified,  complex  controller  with 
verified  switching  logic  (decision  module) 
switches  to  a  verified  safety  controller  in  time  to 
prevent  mishaps.  We  extended  the  architecture  to 
distributed  Simplex  for  DCPS  leveraging  tools 
from  self-stabilizing  distributed  systems. 


sound  translation  framework  from  formal  hybrid  automata  models  to  SLSF  diagrams  (in  particular, 
continuous-time  Stateflow  diagrams,  which  have  behaviors  similar  to  hybrid  automata)  has 
numerous  theoretical  and  practical  challenges.  For  instance,  typical  hybrid  automata  models  do 
not  support  urgency  (although  hybrid  automata  with  urgency  have  been  investigated  recently 
[107]),  while  transitions  in  SLSF  are  urgent  (i.e.,  transitions  are  taken  as  soon  as  they  are  enabled, 
which  is  further  complicated  in  SLSF  due  to  actually  happening  at  zero-crossing  event  points  in 
the  simulation  loop).  SLSF  diagrams  do  not  support  invariants,  while  hybrid  automata  do.  SLSF 
diagrams  (without  stochastic  models)  are  typically  deterministic  (in  both  discrete  transitions  and 
continuous  trajectories),  while  hybrid  automata  are  nondeterministic  (in  both  discrete  transitions 
being  nondeterministic  similar  to  in  nondeterministic  finite- state  automata  [NFAs],  and  continuous 
trajectories  being  described  using  differential  inclusions,  which  allow  for  nondeterministic 
families  of  solutions).  Time-dependent  switching  is  used  to  abstract  more  general  state-dependent 
switching.  Addressing  these  issues  to  ensure  a  notion  of  behavior  preservation  when  translating 
from  hybrid  automata  to  SLSF  (using  an  appropriate  assumption  on  the  behavior  of  the  SLSF 
simulation  loop  and  its  inherently  sampled-time  and  finite-precision  limitations)  to  enable  formal 
guarantees  in  implementations. 


Next,  algorithms  were  developed  for  an  online,  runtime  implementation  of  the  overall 
distributed  emergence  detection  as  candidate  invariants  architecture  depicted  in  Figure  9.  For  the 
dynamic  analysis,  the  specification  inference  methodology  is  implemented  online,  to  infer 
specifications  at  runtime.  Such  methods  have  been  used  for  identifying  security  attacks  in 
ClearView  [102],  but  CPS  have  a  different  set  of  challenges  (real-time,  real  value  approximations, 
etc.)  [51].  For  the  static  analysis,  we  built  upon  preliminary  results  (Figure  13)  for  real-time 
reachability  of  a  single  hybrid  automaton  [7]. 


The  Simplex  Architecture  (see  Figure  11)  ensures  the  safe  use  of  an  unverifiable  complex 
controller  by  using  a  verified  safety  controller  and  verified  switching  logic  [108-113].  This 
architecture  enables  the  safe  use  of  high-performance,  untrusted,  and  complex  control  algorithms 
without  requiring  them  to  be  formally  verified.  Simplex  incorporates  a  supervisory  controller  and 
safety  controller  that  may  take  over  control  if  the  unverified  logic  misbehaves.  The  supervisory 
controller  should  guarantee  the  system  never  enters  an  unsafe  state  (safety),  but  also  use  the 
complex  controller  as  much  as  possible  (minimize  conservatism).  In  preliminary  results  [7],  we 
establish  a  combined  online/offline  approach  that  uses  a  real-time  reachability  computation 
enables  a  proof  of  safety,  but  with  significantly  less  conservatism,  so  the  upgraded  controller  is 
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Figure  12:  Illustration  of  self-stabilization. 
The  DCPS  starts  from  a  set  of  initial  states  Q0 
and  if  it  evolves  over  an  arbitrary  execution 
without  failures  (attacks,  emergent  behavior, 
etc.)  anf,  is  guaranteed  to  self-stabilize  to  a 
set  of  desirable  states  S  where  mission 
progress  is  ensured  and  remain  there.  As  the 
DCPS  operates,  if  failures  occur  and 
executions  ay  are  followed  outside  the  set  S, 
they  are  guaranteed  to  remain  in  a  set  T  that 
at  least  maintains  safety.  Once  failures  stop, 
the  DCPS  again  self-stabilizes  to  S  and  may 
make  mission  progress. 


used  more  frequently  as  in  Figure  13.  In  this  objective,  a 
runtime  assurance  framework  is  developed,  where  the 
safety  controller  is  used  if  distributed  emergent  behavior 
is  detected  online. 

Hynger  has  been  extended  for  runtime  assurance 
tasks  like  detecting  and  thwarting  security  violations  and 
attacks,  similar  to  the  ClearView  tool  that  also  relies  on 
dynamic  analysis  to  detect  changes  in  candidate 
specifications  [51,  102],  Finding  and  monitoring  sets  of 
candidate  invariants  (even  if  not  verified  as  actual 
invariants)  may  be  useful  for  runtime  assurance  and 
resiliency  methods  for  embedded  systems.  If  candidate 
invariants  are  checked  at  runtime  using  a  real-time 
reachability  method  [7],  formal  and  dynamic  runtime 
assurance  may  be  feasible.  Rather  than  purely  sensing 
feedback  in  the  Simplex  decision,  using  changes  in  sets  of 
inferred  candidate  invariants  may  determine  mode 
changes  to  enable  runtime  assurance  in  DCPS. 

3.2. 3.1.  Real-Time  Reachability  for  Networks  of  Hybrid 
Automata 

The  next  research  objective  is  to  extend  real-time 
reachability  to  DCPS  modeled  as  networks  of  hybrid  automata,  which  is  the  first  step  in  developing 
an  RTA  framework  for  DCPS.  Existing  methods  have  only  been  developed  for  a  single  hybrid 
automaton,  so  the  focus  is  on  developing  a  distributed  runtime  verification  method  building  on  the 
real-time  reachability  of  networks  of  hybrid  automata.  This  is  enabled  by  extending  the  symmetry- 
reducing  reachability  framework  for  networks  of  hybrid  automata  [40]  to  those  with  linear  and 
nonlinear  dynamics  and  specifications,  developed  in  Objective  1. 

3.2. 3.2.  RTA  for  Emergence  in  DCPS  with  Distributed  Simplex  and  Self-Stabilization 

Leveraging  both  the  real-time  reachability  for 

hybrid  automata  networks  and  the  Hynger-based 
emergence  monitoring  methods  from  Objective  2,  the 
next  objective  is  to  apply  these  monitoring  methods  in 
RTA  control  of  emergence,  which  is  specified  as 
maintaining  system  state  within  a  given  region  of  the 
state-space  (i.e.,  property  invariance).  We  build  on  the 
theory  and  tools  of  self-stabilizing  distributed  systems 
(see  Figure  12),  which  ensures  eventually  returning  to 
desirable  sets  of  states  in  spite  of  failures,  attacks,  etc. 
Together  with  the  Simplex  RTA  methods,  thus  yields 
the  development  of  a  Distributed  Simplex  RTA 
architecture  for  DCPS.  This  combines  global  and  local 
state  estimation  and  invariant  monitoring.  For 
example,  each  agent  may  deploy  its  own  Simplex 
architecture,  but  what  emergent  behaviors  occur  if  say 
all  agents  start  to  use  fallback  controllers?  What  is  a 
fallback  controller  for  the  entire  distributed  system? 


-0.5  0  0.5  1 

Position,  p  (m) 

Figure  13:  Verifiably  safe  regions  of  state-space 
when  using  complex  controller  for  an  inverted 
pendulum  example,  illustrating  real-time 
reachability’s  advantages  to  offline  verification 
methods  (unverified  simulation  or  LMI-based 
methods  that  yield  ellipsoidal  safe  sets)  in  results 
of  a  Simplex  RTA  framework. 
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These  questions  have  been  addressed  in  our  resulting  publications.  We  leverage  preliminary  results 
[5-7,  52,  53,  1 14]  in  this  direction  to  develop  a  Simplex  architecture  for  emergence  in  DCPS. 

3.2.4.  Evaluating  Analysis,  Monitoring,  and  Control  of  Emergence  in  DCPS  Testbeds 

To  evaluate  the  methods  from  the  previous  objectives,  we  performed  simulations, 
laboratory  experiments,  and  demonstrations  using  a  swarm  of  autonomous  agents,  particularly 
commercially  available  quadrotor  drones.  Typical  safety  properties  that  arise  are  collision 
avoidance  [5,  6,  31-33,  39,  52]  and  convergence  to  some  desired  configuration  and/or  location  [52], 
and  emergent  properties  may  be  consensus,  flocking,  or  unwanted  oscillatory  movements  due  to 
failures,  attacks,  communication  delays,  etc.  Our  results  in  similar  studies  include  verification  of 
autonomous  satellite  maneuvers  [2],  flocking  in  swarm  robotics  in  spite  of  failures  [5,  6],  and  planar 
robotics  [52], 

3.2.4. 1.  Evaluation  of  Emergence  Methods  through  Simulation  Studies 

We  evaluated  the  specification,  verification,  monitoring,  and  control  methods  analytically, 
using  software  tools,  and  in  simulation.  We  extended  the  StarL  framework  that  provides  simulation 
capability  of  DCPS.  Additionally,  StarL  was  used  to  deploy  to  actual  swarm  robot  systems,  so 
altogether  this  enables  evaluation  of  a  correct-by-construction  framework  for  establishing  or 
avoiding  emergence  in  DCPS.  The  StarL  [114,  115]  platform  and  its  offline  simulator  allows  the 
DCPS  to  have  similar  levels  of  concurrency,  asynchrony,  and  other  realistic  effects  as 
implementations.  We  used  the  hybrid  automaton  translation  framework  (Figure  10)  to  convert 
from  formal  models  to  StarL  programs  and  SLSF  diagrams  for  simulation. 

3. 2. 4.1:  Experimental  Evaluation  of  Emergence  Methods  through  Lab  Demonstrations 

We  experimentally  analyzed  the  specification,  verification,  monitoring,  and  control 
methods  for  emergent  behavior  in  DCPS  using  an  indoor  swarm  robotics  system  of  quadrotors. 
This  includes  scenarios  with  emergent  behavior  such  as  flocking,  flocking  in  spite  of  failures  of 
physical,  cyber,  and  communication  components,  and  emergent  behavior  like  collision  avoidance 
that  should  be  invariant.  This  serves  to  validate  the  analytical,  verification,  and  simulation  results, 
and  leverages  the  implementation  of  StarL  programs  [1 14,  1 15]  on  hardware. 

4.  Results  and  Discussion 

4.1.  Key  Results  and  Findings 

The  key  results  and  findings  of  this  project  for  each  objective  are  as  follows. 

Objective  1:  Specification  and  Verification 

The  first  is  in  specifying  behaviors  for  DCPS,  and  this  resulted  in  the  creation  of  a  novel 
formal  specification  language  called  hyperproperties  for  signal  temporal  logic  (HyperSTL),  which 
arguably  is  the  most  complete  specification  language  for  formally  describing  behaviors  of  DCPS 
[36].  Also  for  specification  of  behavior,  the  perspective  of  considering  cyber,  physical,  and  cyber¬ 
physical  specifications  in  DCPS  is  a  key  insight  [35].  The  second  is  in  addressing  the  state-space 
explosion  problem  for  DCPS,  particularly  through  the  use  of  order-reduction  [116]. 

Objective  2:  Monitoring 

For  monitoring  DCPS  behavior,  the  runtime  monitoring  framework  built  using  Hynger  to 
check  if  behaviors  observed  at  runtime  is  the  key  result.  By  monitoring  whether  specifications  may 
be  violated  at  runtime  gives  an  indication  that  emergent  behavior,  or  some  other  anomalous 
behavior,  may  be  occurring  [51]. 
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Objective  3:  Control 

The  perspective  of  runtime  assurance  using  the  Simplex  architecture  seems  particularly 
powerful  for  mitigating  emergent  behavior,  if  it  is  undesirable  [117].  This  approach  may  be 
impactful  and  useful  when  artificial  intelligence  (AI)  and  machine  learning  (ML)  components  are 
incorporated  in  DCPS,  and  this  direction  of  research  was  a  key  outcome. 

Objective  4:  Evaluation 

The  methods  above  were  evaluated  in  several  case  studies,  particularly  within  the 
distributed  robotics  framework  of  StarL.  Videos  of  scenarios  are  included  with  the  deliverables. 
Numerous  benchmark  case  studies  were  published  [118-122]. 

4.2.  Related  Work 


No:  bug 


[Yes:  proof 
Figure  14:  General  formal 
verification  problem. 


Model-based  verification  typically  develops  a  model  of  a  system  and  properties 
(specifications)  are  (manually,  semi-automatically,  or  automatically)  checked  for  that  model. 
However,  most  safety  issues  induced  by  software  bugs  are  not  a  result  of  design  errors,  but  are  the 
result  of  implementation,  reuse,  upgrade,  and  maintenance  errors.  While  a 
priori  model-based  design  (MBD)  and  clean-slate  approaches  like 
DARPA’s  HACMS,  seL4,  Bedrock,  and  CompCert  [123-127]  are  of 
critical  important  and  especially  useful  for  subcomponent  verification, 
most  systems  being  designed  today  utilize  a  development  process  where 
engineers  write  software  and  systems  are  integrated  from  numerous 
components.  Additionally,  while  there  are  many  standards  to  help 
improve  CPS  safety  in  various  domains  (like  ISO  26262  functional  safety 
standard  for  road  vehicles  [128]  and  MISRA  C  [129]),  as  CPS  have  exponential  gains  in  software 
embedded  in  them,  these  reliability  problems  will  only  become  exacerbated  [8,  9].  Rare  cyber¬ 
physical  failure  scenarios  and  distributed  emergence  motivate  runtime  contingencies  to  assure 
safe,  if  degraded,  operation. 

Dynamic  Specification  Inference :  There  are  many  benefits  of  dynamic  analysis  such  as  using 
implementations  instead  of  models  [81,  82,  130]  to  find  dynamic  program  specifications  [130].  The 
limitation  is  results  are  unsound  without  additional  reasoning.  Finding  specifications  of  systems  is 
a  maturing  field  within  software  engineering  [81,  82,  130-133],  and  recent  simulation-based 
approaches  in  hybrid  systems  and  CPS  like  those  used  in  S-TaLiRo,  Breach,  and  C2E2  can  be 
viewed  as  dynamic  analysis  [91,  97,  134-139].  Invariants  are  properties  of  a  system  that  always 
hold,  while  conditional  invariants  may  hold  at  certain  program  points,  for  example,  at  the 
beginning  or  end  of  a  function  call  (pre/post  conditions).  Daikon  has  found  candidate  invariants  of 
hybrid  models  of  biological  system  [140]  and  distributed  systems  [141,  142],  and  this  illustrates  a 
proof-of-concept  to  use  it  for  hybrid  systems.  Alternative  approaches  analyze  simulation  traces 
from  complex  Matlab/Simulink  models  [97,  114,  138,  139],  but  require  a  priori  specifications  or 
require  templates  from  restricted  classes  of  logic. 

Verification  of  Hybrid  Systems:  Formal  verification  aims  to  solve  the  problem  posed  in  Figure 
14:  does  a  given  formal  system  model  (often  an  automaton)  <A  satisfy  a  given  specification 
(property)  P?  Automated  formal  verification  (as  instantiated,  for  example,  in  model  checkers), 
aims  to  develop  an  algorithm  to  solve  the  formal  verification  problem,  instead  of  using  semi- 
automated  methods  such  as  interactive  theorem  provers.  A  hybrid  automaton  [30,  31,  84,  143,  144] 
is  a  formal  model,  and  is  essentially  a  finite-state  machines  with  additional  continuous  variables 
that  may  evolve  according  to  ordinary  differential  equations  (ODEs)  or  inclusions  that  may  differ 
in  each  state.  Hybrid  automata  provide  a  formal  mathematical  semantics  for  formal  verification  of 
properties  specified  in  some  formal  language  using  many  techniques  [31,  76-78,  144].  While 
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automated  formal  verification  is  undecidable  for  many  interesting  classes  of  systems  (such  as 
general  software  or  general  hybrid  automata),  numerous  advances  have  been  made  in  the  past  few 
decades.  Explicit-state  and  symbolic  model  checking  [68,  145-149]  is  common  place  in  numerous 
industrial  semiconductor  development  processes,  aided  in  part  by  automata-theoretic  advances  and 
developments  like  efficient  representations  like  BDDs  [150],  DDDs  [151],  SAT-encodings  [152, 
153],  and  recently,  SMT-encodings  [94,  95,  154-157]  and  quantified  encodings  [158].  Embedded 
and  hybrid  systems  have  likewise  benefited  from  advances  such  as  those  implemented  in  HyTech 
[78],  KRONOS  [159],  Charon  [160],  Checkmate  [161],  the  ellipsoidal  toolbox  [162],  PHAVer  [76], 
KeYmaera  [163, 164],  SpaceEx  [77, 165],  and  simulation-based  verification  [134-136, 139,  166-168]. 
SMT-based  techniques  have  been  used  for  reachability  analysis  of  hybrid  systems  in  SAT-modulo- 
ODEs  [79,  169-171],  and  for  automatically  discharging  deductive  proofs  of  safety  by  inductive 
invariance  in  Passel  [31,  32,  38-40]. 

Translating  Hybrid  Automata  to  Implementations'.  Efforts  have  recently  been  investigated  for 
translating  timed  automata  to  Mathworks  Simulink/Stateflow  ( SLSF )  diagrams,  such  as  UPP2SF 
that  converts  UPPAAL’s  timed  automata  to  SLSF  diagrams  while  maintaining  certain  properties 
of  executions  (i.e.,  a  form  of  soundness)  [172-175].  A  vast  amount  of  existing  work  exists  in  the 
opposite  direction,  of  translating  from  SLSF  to  formal  models  like  hybrid  automata,  extended  finite 
state  machines  (EFSMs),  etc.  and  between  hybrid  systems  formalisms  [176-187].  However,  tools 
translating  from  SLSF  are  impractically  difficult  to  build,  in  part  since  SLSF  does  not  have  a 
formal  semantics  (although  efforts  have  tried  to  define  some  [188,  189]),  and  commercially 
available  tools  such  as  Ansys/Esterel’s  SCADE  Lustre  [190]  converter  tool  (that  translates  SLSF 
to  Lustre  with  precise  semantics)  are  not  only  impracticably  large  to  build  in  an  academic  setting 
(e.g.,  Esterel’s  converter  has  millions  of  lines  of  code),  but  are  theoretically  unsound,  albeit  very 
useful.  Additionally,  all  commercially  viable  converters  only  support  discrete  SLSF  diagrams  (or 
discretizations  thereof),  and  may  not  include  continuous-time  blocks  like  continuous-time 
Stateflow  diagrams  [183,  185,  191].  Academic  efforts  exist  to  translate  from  SLSF  to  hybrid 
models  (such  as  HyLink  [192-194]  and  others  [176,  184]),  but  the  vast  effort  in  creating  viable 
translators  make  it  impractical. 

4.3.  Deliverables 


Table  1  describes  the  deliverables  produced  through  this  project,  which  includes  quarterly 
status  reports,  final  technical  reports,  software  deliverables  including  source  code  and  prototypes, 
and  APIs  including  documentation. 


Objective 

Deliverables 

Objective  1: 

Modeling  and 
Analysis 

HyST/Passel  software  tool,  with  extensions  to  linear  and  nonlinear  local 
dynamics  allowing  modeling  of  the  swarm  robotics  DCPS  case  study  and 
emergent  properties  in  general  DCPS.  Software  deliverables  with  source  code 
and  prototypes,  and  APIs  including  documentation. 

Online:  httDs://sithub.com/verivital/hvst 

Objective  2: 

Monitoring 

Hynger  invariant  inference  software  tool  for  distributed  emergence 
monitoring.  Software  deliverables  with  source  code  and  prototypes,  and  APIs 
including  documentation. 

Online:  httns://bitbucket.org/verivital/hvnger 
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Objective  3: 

Control 

Real-time  reachability  tool  and  RTA  framework  for  emergence  in  DCPS 
relying  on  self-stabilization.  Software  deliverables  with  source  code  and 
prototypes,  and  APIs  including  documentation. 

Online :  http s : //bitbucket .or g/veri vital/rtreach 

Objective  4: 

Evaluation 

Models  of  the  swarm  robot  case  studies;  source  code  for  the  control  software; 
source  code  and  design  files  for  the  overall  swarm  robot  evaluation  system. 
Software  deliverables  with  source  code  and  prototypes,  and  APIs  including 
documentation. 

Online :  httns  ://github  .com/verivital/starl 

Videos:  httr>s://www.  youtube. com/channel/UCl-RPioacWVNLOKiuxrbn9  A 

Table  1:  Deliverables  for  analysis,  monitoring,  and  control  of  emergence  in  DCPS. 


5.  Conclusion 

This  project  studied  emergent  behavior  in  DCPS  by  developing  formal  specification 
languages,  formal  verification  methods  within  the  HyST  software  tool,  heuristic -based  runtime 
monitoring  within  the  Hynger  software  tool,  and  Simplex-based  runtime  assurance.  Together,  the 
project  demonstrates  the  capability  to  detect,  monitor,  and  control  emergent  behavior  in  DCPS. 
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Abstract — Cyber-physical  systems  (CPS)  consist  of  physical 
entities  that  obey  dynamical  laws  and  interact  with  software 
components.  A  typical  CPS  implementation  includes  a  discrete 
controller,  where  software  periodically  samples  physical  state  and 
produces  actuation  commands  according  to  a  real-time  schedule. 
Such  a  hybrid  system  can  be  modeled  formally  as  a  hybrid 
automaton.  However,  reachability  tools  to  verify  specifications 
for  hybrid  automata  do  not  perform  well  on  such  periodically- 
scheduled  models.  This  is  due  to  a  combination  of  the  large 
number  of  discrete  jumps  and  the  nondeterminism  of  the 
exact  controller  start  time.  In  this  paper,  we  demonstrate  this 
problem  and  propose  a  solution,  which  is  a  validated  abstraction 
mechanism  where  every  behavior  of  the  original  sampled  system 
is  contained  in  the  behaviors  of  a  purely  continuous  system  with 
an  additive  nondeterministic  input.  Reachability  tools  for  hybrid 
automata  can  better  handle  such  systems.  We  further  improve 
the  analysis  by  considering  local  analysis  domains.  We  automate 
the  proposed  technique  in  the  Hyst  model  transformation  tool, 
and  demonstrate  its  effectiveness  in  a  case  study  analyzing  the 
design  of  a  yaw-damper  for  a  jet  aircraft. 

I.  Introduction 

Periodic  real-time  scheduling  is  a  widespread  method  used 
to  control  a  physical  plant  as  part  of  a  cyber-physical  system 
(CPS).  Typical  schedulers,  such  as  rate-monotonic  (RM)  or 
earliest  deadline  first  (EDF)  [1],  give  a  guarantee  of  peri¬ 
odic  execution.  In  each  period,  sensors  are  read,  the  control 
algorithm  is  run,  and  actuator  outputs  are  set.  The  physical 
world,  on  the  other  hand,  evolves  continuously.  Models  of  the 
physical  world  may  be  given  using  differential  equations  that 
are  obeyed  at  all  times. 

In  this  work,  we  analyze  the  periodically-scheduled  con¬ 
troller  subsystems  of  CPS  using  hybrid  automata  [2]  and 
associated  analysis  tools.  A  hybrid  automaton  can  directly 
model  both  the  continuous  behaviors  and  discrete  aspects 
that  arise  when  real-time  scheduling  and  sampled  control  is 
combined  with  a  continuously-evolving  physical  plant.  The 
set  of  reachable  states  of  a  hybrid  automaton,  if  it  can  be 
computed  or  overapproximated,  can  be  used  to  formally  prove 
control-theoretic  properties  about  the  system’s  transient  and 
steady-state  behavior.  The  controller  subsystem  models,  after 
being  proven  correct,  could  then  be  integrated  with  hybrid 
automaton  models  of  other  parts  of  the  system  using  modeling 
methods  like  hybrid  input/output  automata  (HIOA)  [3].  Rea¬ 
soning  about  properties  of  the  combined  system  could  then  be 
performed  using  assume-guarantee  reasoning  [4].  With  such 
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hybrid  systems  analysis,  properties  can  be  formally  proven 
about  sets  of  initial  states  as  well  as  behaviors  under  bounded 
sensor  error,  actuator  error,  and  other  uncertainties.  This  has 
the  potential  to  detect  errors  not  found  during  simulation 
and  testing,  which  deal  with  single  initial  states  and  specific 
execution  traces. 

Directly  analyzing  the  controller  subsystems  of  CPS  using 
hybrid  automaton  reachability  tools,  unfortunately,  does  not 
usually  work.  One  issue  is  that  a  large  number  of  controller 
updates  need  to  be  considered  in  the  analysis.  The  control  code 
may  need  to  be  run  tens  or  hundreds  of  times  a  second,  and 
the  physical  system  may  need  to  evolve  for  tens  of  seconds 
to  show  the  properties  of  interest.  The  number  of  discrete 
transitions  that  occur  thus  becomes  extremely  large.  Real-time 
schedulers  may  also  have  variability  in  the  exact  scheduling 
time  of  the  controller.  Hybrid  automaton  reachability  analysis 
tools  perform  poorly  in  such  cases,  with  error  bounds  growing 
unacceptably  large  in  the  presence  of  many  discrete  transitions 
and  timing  uncertainty  [5],  [6], 

In  order  to  overcome  these  challenges,  we  apply  a  variant  of 
the  continuization  technique  [7],  where  a  fast-switching  hybrid 
system  is  abstracted  by  a  continuous  system  with  an  additive 
nondeterministic  input.  We  provide  theoretical  methods  to 
compute  bounds  on  the  nondeterminism  input  needed  for  the 
continuization  of  periodically-scheduled  controllers,  which  is 
essential  for  abstraction  soundness.  The  developed  approach  is 
then  automated  using  the  Hyst  [8]  model  transformation  tool. 
In  this  way,  we  provide  both  a  theoretical  method  that  enables 
controller  analysis  with  hybrid  automaton  reachability  tools, 
and  a  practical  way  to  use  it. 

The  main  contributions  of  this  paper  are: 

•  the  modeling  of  periodically-controlled  CPS  using  hybrid 
automata,  with  several  models  proposed  based  on  possi¬ 
ble  implementation  variations, 

•  the  validated  use  of  continuization  to  enable  the  analysis 
of  these  models,  and  a  theoretical  method  to  compute  the 
bound  on  the  nondeterminism  globally  as  well  as  within 
local  analysis  domains, 

•  the  implementation  of  the  proposed  technique  in  the  Hyst 
model  transformation  tool,  which  allows  rapid  application 
to  new  hybrid  automaton  models,  and 

•  a  demonstration  of  the  effectiveness  of  the  proposed 
analysis  approach  on  the  design  of  a  yaw  damper  system 
for  a  747  jet  aircraft. 
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In  the  next  section,  we  present  a  brief  background  on  mod¬ 
eling  hybrid  systems,  give  direct  approaches  for  modeling  real- 
time-scheduled  controllers  with  hybrid  automata,  and  provide 
reachability  results  showing  scalability  issues  with  these  direct 
models.  Next,  Sec.  Ill  describes  continuization  and  methods 
for  computing  the  nondeterministic  bounds  it  uses,  which  are 
essential  for  method  accuracy.  Then  Sec.  IV  briefly  describes 
the  Hyst  model  transformation  tool  and  the  illustrates  the 
continuization  pass  that  implements  the  technique  developed  in 
this  paper.  Sec.  V  provides  a  case  study  showing  the  advantage 
of  the  analysis  on  a  yaw  damper  control  system  for  a  747 
aircraft.  A  brief  discussion  of  related  techniques,  especially  a 
comparison  versus  classical  control  theoretic  methods  is  given 
in  Sec.  VI,  followed  by  a  conclusion. 

II.  Hybrid  Systems  Modeling 

The  controller  subsystem  of  a  cyber-physical  system  (CPS) 
consists  of  a  physical  system  interacting  with  a  software 
controller,  running  periodically  on  a  system  using  a  real-time 
scheduler.  A  specific  implementation  can  be  formalized  using 
a  hybrid  automaton  model,  and  then  its  behavior,  as  well  as 
the  behavior  of  a  composition  of  these  subsystem  models, 
can  be  analyzed  using  hybrid  automata  reachability  tools.  In 
this  section,  we  elaborate  on  modeling  controller  subsystems 
using  the  hybrid  automaton  formalism.  We  first  review  hybrid 
automata  (Sec.  II-A),  then  propose  three  models  that  capture 
different  possible  implementations  of  a  controller  subsystem 
of  a  CPS  (Sec.  II-B).  Finally,  we  attempt  to  directly  perform 
reachability  analysis  of  these  systems  using  reachability  anal¬ 
ysis  tools  (Sec.  H-C),  which  is  shown  to  be  challenging. 

A.  Preliminaries 

A  hybrid  automaton  is  a  formal  model  that  captures  both 
discrete  behaviors  as  well  as  continuous  dynamics  present  in 
a  hybrid  system.  Roughly,  it  is  a  finite  state  machine  with 
ordinary  differential  equations  defined  in  each  mode  for  a  set 
of  real-valued  continuous  variables. 

Definition  1  (Hybrid  Automaton).  A  Hybrid  Automaton  is  a 
tuple 

TL  =  (Loc,  Var,  Init,  Flow,  Trans,  Inv)  that  defines: 

•  a  finite  set  of  locations  Loc, 

•  a  set  of  n  real-valued  continuous  variables  Var  = 
{x\,  ■  ■  -,xn}, 

•  an  initial  condition  Init  C  Rn  for  each  t  €  Loc, 

•  for  each  location  I,  a  relation  Flow(£)  relating  variables 
and  their  derivatives, 

•  a  set  of  discrete  transitions  Trans,  where  each  element  is 
a  tuple  (£,g,r,£')  with  source  location  £,  guard  g  given 
as  constraint  on  K",  reset  r  given  as  a  function  from 
Wnto  Rn,  and  destination  location  U , 

•  an  invariant  Inv(f)  C  R"  for  each  location  L 

A  state  of  a  hybrid  system  is  a  tuple  (£,X),  where  the 
discrete  state  is  £  £  Loc  and  the  continuous  state  X  is  a 
valuation — a  mapping  from  a  variable  name  to  a  point  in  the 
reals — of  the  continuous  variables  in  Var. 


Definition  2  (Trajectory).  A  trajectory  of  a  hybrid  system  is 
an  alternating  sequence  of  continuous  evolutions  and  discrete 
transitions,  starting  from  a  state  in  Init.  Trajectories  are 
subject  to  the  following  restrictions: 

•  the  first  state  of  the  trajectory  is  an  element  of  Init, 

•  during  each  continuous  evolution,  the  continuous  state 
evolves  over  an  interval  of  real-valued  time  in  accordance 
with  the  differential  equations  defined  by  Flow, 

•  during  each  continuous  evolution,  the  continuous  states 
always  satisfy  the  location’s  invariant1,  and 

•  during  each  discrete  transition,  the  prestate  is  contained 
in  transition ’s  guard,  and  the  change  in  state  corresponds 
to  applying  the  reset  function  to  the  continuous  prestate 
and  updating  the  location  to  £' . 

Definition  3  (Reachable  Set).  The  set  of  all  states  that  exist  in 
any  trajectory  is  called  the  reachable  set.  For  a  given  hybrid 
automaton  PL,  we  use  REACH("H)  to  denote  the  reachable  set 
of  PL.  Given  a  subset  of  the  variables  Y  C  Var  of  hybrid 
automaton  PL,  the  reach  set  projected  onto  those  variables  is 
written  as  REACH("H)  j.  Y.  Typically  we  will  be  concerned 
with  time-bounded  reachable  sets,  where  the  amount  of  time 
that  has  elapsed  during  the  continuous  evolution  portions  of 
each  trajectory  is  less  than  or  equal  to  some  given  bound. 

B.  CPS  Modeling 

We  now  describe  three  different  ways  that  a  CPS  con¬ 
troller  subsystem  can  be  modeled  using  the  hybrid  automaton 
formalism,  which  correspond  to  different  possible  system 
implementations.  First,  we  introduce  the  notion  of  a  Sampled 
CPS ,  which  has  a  continuous  portion  governed  by  differential 
equations,  and  a  controller_update  function  that  updates  the 
discretely-controlled  variables. 

Definition  4  (Sampled  CPS).  A  Sampled  CPS  is  a  system 
with  n  continuous  variables  divided  into  two  groups.  The 
first  np  <  n  variables  are  the  physical  variables,  and  the 
remaining  nc  =  n  —  np  variables  are  the  cyber  variables. 
The  set  of  variables  Var  =  {xi,X2,  ■  ■  ■  ,xn}  is  partitioned 
into  physical  variables  Xp  =  {pi,p2,  ■  ■  ■  ,pnp}  txnd  cyber 
variables  Xc  =  {ci,  C2, . . . ,  cncj,  where  each  variable  Xi  €  R. 
Each  physical  variable  has  an  associated  differential  equa¬ 
tion,  p-|  =  fi,  p2  =  /a,  •  ■  •,  Pnp  =  fnp,  where  each 
Pi  =  fi  is  a  function  R"  — >  R.  To  ensure  existence  and 
uniqueness  of  the  solutions,  the  differential  equations  are 
assumed  to  be  Lipschitz  continuous  in  the  domain  of  interest. 
The  dynamics  for  the  physical  variables  are  provided,  so 
Fp  =  (/i,  /2,  •  •  • ,  fnp)  is  given.  The  remaining  nc  variables 
are  set  periodically  in  control  software,  and  remain  constant 
betw’een  updates  (zero-order  hold).  Their  differential  equa¬ 
tions  are  given  as  t\  =  0,  £2  =  0,  . . ccn  =  0.  The 
control  software  is  defined  by  a  function  controller_update  : 
R"  — >  R"c,  which  updates  the  cyber  variables  based  on  the 

'if  at  some  point  the  invariant  were  to  become  false,  a  discrete  transition 
must  be  taken  immediately.  If  no  transition’s  guards  are  enabled,  the  model 
is  said  to  deadlock  as  time  cannot  advance. 
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Guard:  c  =  T? 

Xc  :=  controller _update(Xp,  Xc) 
c  :=  0 


Guard:  c  =  T? 

Xc  :=  controller, _update(Xs,Xc) 
Xs  :=  Xp 
c  :=  0 


Guard:  True? 


(c)  Model  3 


Fig.  1:  Various  hybrid  automaton  models  formalize  different  implementations  of  a  periodically-sampled  CPS. 


system  state.  The  controller  update  function  can  be  decom¬ 
posed  into  nc  functions  where  each  one  updates  a  single 
cyber  variable,  C\  :=  controller_update1(Xp,  A'c), . . . ,  cnc  := 
controller_updatenc(Xp,  Xc).  In  this  work,  we  will  restrict  the 
controller  update  functions  to  ones  that  are  differentiable  and 
locally  Lipschitz  continuous  in  the  input  arguments,  in  the 
domain  of  interest  (for  example,  discrete  approximations  of 
continuous  controllers). 

Model  1:  The  simplest  model  is  for  a  strict  periodic 
controller,  where  the  control  software  runs  with  a  given  period, 
T.  This  could  correspond  to  a  system  using  a  time-division 
multiple-access  (TDMA)  or  other  time-triggered  scheduler, 
where  the  control  task  is  nonpreemptive  and  the  worst-case 
execution  time  (WCET)  fairly  short.  In  the  model,  a  single 
location  exists  where  time  can  elapse.  An  extra  clock  variable, 
c,  is  added  to  the  hybrid  automaton  that  ticks  at  rate  one 
(c  =  1).  When  the  clock  reaches  the  period,  a  transition  is 
forced  by  an  invariant  in  the  single  location  that  c  <  T, 
which  prevents  continuous  evolutions  from  continuing.  The 
transition  executes  the  controller  logic  when  the  clock  reaches 
the  period,  then  resets  the  clock  to  0,  and  subsequently  repeats 
periodically.  A  hybrid  automaton  visualization  of  this  model 
is  shown  in  Fig.  1(a).  The  strict  periodic  controller,  however, 
does  not  exactly  capture  the  behavior  of  a  system  using  a 
real-time  scheduler.  A  scheduler  like  rate-monotonic  (RM)  or 
earliest  deadline  first  (EDF),  provides  a  guarantee  of  execution 
at  some  point  within  the  period. 

Model  2:  An  alternative  implementation,  which  uses  a  real¬ 
time  scheduler  such  as  RM  or  EDF  would  sample  the  system 
at  the  start  of  the  period,  and  write  the  actuation  values  at 
the  end  of  the  period.  This  can  be  modeled  using  a  hybrid 
automaton  by  starting  with  the  strictly  periodic  system  (Model 
1)  and  adding  np  additional  cyber  variables,  which  we  call  A's, 
with  derivatives  equal  to  zero  that  model  the  sampled  state. 
On  the  actuator  assignment  (controller _update)  at  the  end  of 
each  period,  the  controller  logic  will  then  compute  on  the  state 
sampled  from  the  start  of  the  period.  After  updating  the  cyber 
variables,  the  physical  state  would  then  be  sampled  again  and 
stored  into  Xs  for  use  at  the  end  of  the  next  period.  The 
hybrid  automaton  model  of  this  system  is  given  in  Fig.  1(b). 
The  downside  of  such  a  controller  is  there  is  a  one  period 


delay  introduced  into  the  system,  which  may  affect  control 
performance,  as  well  as  np  additional  variables  in  the  model, 
which  may  affect  analysis  scalability. 

Model  3:  An  alternative  implementation  may  consider 
directly  sampling  and  actuating  at  some  point  during  each 
period,  where  the  sampling  point  is  nondeterministic.  This 
would  be  a  reasonable  model  if  the  control  task’s  execution 
is  short  and  the  task  is  non-preemptive.  This  model  is  similar 
to  the  strictly  periodic  Model  1  (Fig.  1(a)),  except  that:  (1) 
a  second  mode  is  added  to  indicate  if  the  controller  has 
run  yet  during  the  period,  (2)  the  first  transition  (the  call 
to  controller_update)  happens  nondeterministically  up  to  the 
period  T  owing  to  the  invariant  c  <  T,  and  (3)  the  second 
transition  (the  end  of  the  control  period)  happens  when  the 
clock  reaches  T  time.  The  modified  automaton  is  shown 
in  Fig.  1(c).  This  model  uses  nondeterminism  in  discrete 
transitions  to  capture  the  type  of  guarantee  provided  by  a  real¬ 
time  scheduler:  that  the  control  logic  will  execute  and  finish 
at  some  point  within  each  period. 

More  complicated  models  could  also  be  considered.  For  ex¬ 
ample,  if  the  execution  time  was  non-negligible  or  the  task  was 
preemptive,  the  state  could  be  sampled  nondeterministically 
at  some  point  during  the  period  minus  the  WCET,  and  then 
actuation  could  be  performed  nondeterministically  up  to  the 
end  of  the  period. 

C.  Preliminary  Reachability  Analysis 

Although  hybrid  automata  can  model  real-time  scheduled 
controllers  and  plants  as  shown  above,  an  important  factor 
is  tractability  of  analysis.  Since  analysis  of  even  moderately- 
complicated  hybrid  automata  is  undecidable  [9],  tools  often 
compute  an  overapproximation  of  the  reachable  states,  which 
is  sufficient  for  safety  analysis  (making  sure  unsafe  states  are 
not  reachable).  If  the  set  of  reachable  states  may  be  computed 
for  unbounded  time  (if  the  reachability  algorithm  reaches  a 
fixed-point)  and  the  resulting  set  of  states  is  bounded,  then 
conclusions  can  also  be  drawn  about  system  stability.  In  the 
presence  of  a  large  number  of  discrete  switches,  reachability 
analysis  tools  may  significantly  overapproximate  the  reachable 
set  of  states,  due  to  the  need  to  perform  intersections  of 
reachable  sets  with  surfaces  representing  guard  conditions  [6]. 
These  intersections  are  typically  done  geometrically,  and  result 
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(a)  Simulations  from  x  €  {0,0.1}  (bj  SpaceEx  Reachability  (c)  Flow*  Reachability 

Fig.  2:  The  response  for  the  periodically-controlled  double-integrator  system  from  Example  1  converges  in 
simulation,  but  appears  to  diverge  during  reachability  analysis. 


in  an  overapproximation  of  the  actual  intersection,  introducing 
some  error  at  every  discrete  transition.  Due  to  this  concern,  we 
empirically  evaluate  the  performance  of  two  modern  reacha¬ 
bility  tools,  SpaceEx  [5]  and  Flow*  [10],  [11],  on  a  simple 
control  system  using  the  approach  from  Model  1  (Fig.  1(a)). 

Example  1  (Double-Integrator  System).  A  double-integrator 
system,  such  as  point  moving  along  a  1-d  line  controlled 
through  its  acceleration,  has  two  physical  variables:  x,  its 
position,  v,  its  velocity,  and  a  single  cyber  variables  a,  its 
acceleration.  The  dynamics  are  x  =  v,  v  =  a,  and  the 
acceleration  a  is  set  periodically  by  the  control  logic.  There  is 
a  fixed  setpoint  the  system  tries  to  move  towards  at  x  =  1.  The 
acceleration  is  set  using  a  PD  controller  with  gains  P  =  10 
and  D  =  3.  The  controller_update  function  periodically 
assigns  a  :=  P  *  (1  —  x)  +  D  *  —v.  The  period  of  the  control 
task  is  T  =  0.005  seconds  (200  Hz).  The  initial  states  are 
x  £  [0,  0.1]  and  v  =  0. 

Using  the  system  in  Example  1,  we  construct  the  corre¬ 
sponding  hybrid  automaton  (shown  in  the  Appendix  in  Fig.  8) 
and  examine  the  controller’s  response.  A  control  Lyapunov 
function  may  be  derived  to  show  stabilization  of  the  purely 
continuous  system  to  the  setpoint  of  x  =  1  and  v  =  0.  In 
Matlab  simulations  of  the  periodically-sampled  system  from 
the  boundary  of  the  initial  states  (from  both  x  =  0  and 
x  =  0.1),  the  system  easily  converges  to  the  setpoint.  When 
performing  reachability,  however,  both  SpaceEx  and  Flow* 
produce  divergent  reachable  sets,  due  to  overapproximation 
error  introduced  at  each  of  the  discrete  transitions.  The  simu¬ 
lations  and  reachability  visualization  are  shown  in  Fig.  2. 

Although  effort  was  taken  to  optimize  various  tool  parame¬ 
ters,  they  could  likely  be  further  adjusted  to  get  a  slightly  better 
response.  For  this  particular  system,  if  the  tools  had  built-in 
support  for  time-triggered  transitions  and  could  infer  that  the 
clock  acts  as  a  time-trigger  for  the  discrete  transition,  the  error 
in  the  computation  could  likely  be  reduced  (although  we  could 
not  find  time-triggered  support  in  either  tool’s  documentation). 
However,  this  would  not  work  for  the  nondeterministic  switch 
in  Model  3  (Fig.  1(c)),  since  that  discrete  transition  (invocation 
of  controller _update)  can  occur  at  any  time  within  the  period, 
based  on  the  guarantees  provided  by  schedulers  like  RM  and 


EDF.  The  problem  of  accumulated  error  in  reachability  from 
many  discrete  transitions,  in  general,  cannot  be  eliminated. 

III.  CONTINUIZATION  FOR  IMPROVED  ANALYSIS 

The  occurrence  of  many  discrete  transitions  leads  to  accu¬ 
mulated  error  during  reachability  analysis  because  of  a  need 
to  repeatedly  take  intersections  of  sets  of  states  with  the 
transition  guards.  One  idea  to  get  better  accuracy,  therefore, 
is  to  eliminate  the  discrete  transitions  altogether.  Intuitively, 
this  process  relies  on  the  observation  that  the  behavior  of  the 
periodically-sampled  system  is  contained  in  the  behavior  of  the 
continuously-controlled  system  with  some  additional  bounded 
nondeterministic  input. 

This  process  of  validated  abstraction  of  the  sampled  hybrid 
automaton  by  a  continuous  one  is  called  continuization  [12], 
and  is  briefly  reviewed  in  the  next  subsection  (Sec.  III-A). 
Here,  we  apply  the  continuization  idea  in  order  to  analyze 
periodic  control  systems,  which  has  not  been  done  before.  This 
process  relies  on  having  a  bound  on  the  speed  of  changes  of  the 
cyber  variables,  and  computing  this  bound  is  then  described 
(Sec.  III-B). 

A.  Continuization 

Continuization  is  the  process  of  abstracting  a  system  with 
many  discrete  switches  by  a  continuous  one  with  an  extra 
nondeterministic  input.  Previously,  it  was  used  to  analyze 
rapidly-switching  electric  circuits  [12],  specifically  locking 
time  and  stability  properties  for  charge-pump  phase-locked 
loops.  The  key  challenge  when  performing  continuization 
is  determining  the  amount  of  nondeterministic  input  that  is 
necessary  in  order  to  guarantee  that  all  behaviors  of  the 
sampled  system  are  captured  by  the  continuous  one,  but  not 
too  much  that  analysis  accuracy  suffers. 

In  the  earlier  circuit  work,  this  was  done  by  solving  for 
the  change  of  state  in  one  cycle  with  a  known  switching  time. 
Since  there  was  no  closed-form  solution  for  the  switching  time, 
interval  analysis  was  performed  using  the  ranges  of  possible 
switching  times,  and  then  this  was  used  to  derive  conservative 
bounds  on  the  change  in  state. 

We  want  to  apply  continuization  in  order  to  analyze 
periodically-controlled  CPS.  We  formalize  this  process  by 
using  sampling  deviation  functions. 
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Definition  5  (Sampling  Deviation).  A  sampling  deviation  uji 
is  a  function  ffi.  — >  K  x  I,  which,  given  a  time,  produces  an 
upper  and  lower  bound  on  the  difference  of  a  cyber  variable 
c.i  £  Xc,  between  its  value  in  a  sampled  CPS  and  the  update 
function  controller_update(Xp ,  Xc). 

Given  a  sampling  deviation  function  lu,  for  each  cyber  vari¬ 
able,  we  can  construct  an  overapproximation  of  the  sampled 
CPS.  First,  we  construct  a  continuous  approximation  of  the 
sampled  CPS. 

Definition  6  (Continuous  Approximation).  A  continuous  ap¬ 
proximation  of  a  sampled  CPS  is  a  hybrid  automaton  where 
the  controller  logic  is  run  continuously.  That  is,  the  discrete 
update  for  each  cyber  variables  in  Ci  £  Xc  is  removed  from 
the  system,  and  each  cyber  variable’s  differential  equation  is 
set  to  Ci  =  ^controller  update i(Xp,  Xf).  The  variable  cfs 
initial  value  is  set  to  the  value  when  the  controller  is  run  at 
the  original  initial  state,  controller_updatei(Xp(0),  Xc{0)). 

The  continuous  approximation  differs  from  the  original 
sampled  CPS.  A  continuized  abstraction  accounts  for  this 
difference  by  adding  nondeterminism  to  every  occurrence  of 
each  cyber  variable  within  the  continuous  approximation. 

Definition  7  (Continuized  Abstraction,  Continuization).  A 
continuized  abstraction  PLC  of  a  sampled  CPS  PL  is  constructed 
starting  from  PL’s  continuous  approximation.  Each  occurrence 
of  a  cyber-variable  Ci  in  the  continuous  approximation  gets  an 
extra  term  added  equal  to  the  sampling  deviation  uj1.  If  any 
of  the  u>i  change  over  time,  an  additional  time  variable  t  is 
added  to  the  system  that  starts  at  0  and  ticks  at  rate  1  forever. 

The  model  constructed  using  the  above  continuization  ap¬ 
proach  will  have  trajectories  of  the  physical  variables  that 
contain  all  the  behaviors  in  the  original  sampled  CPS. 

Theorem  1  (Soundness  of  Continuization).  Given  a  sampled 
CPS  PL  as  well  as  its  continuized  abstraction  PLC,  REACH  (PL)  j, 
Xp  C  reach("Hc)  |  Xp. 

Proof.  Consider  any  cyber  variable  c7;  £  Xc.  Let  Valsampie(j(ci) 
be  the  value  of  the  variable  in  the  sampled  CPS,  and 
Valabstract (Ci)  be  the  value  of  the  variable  in  the  continuized 
abstraction.  At  any  time  t  in  a  trajectory,  we  first  show  that 
'ClI  s71|np|ed  (C’j )  £  Valabstract(G)  T  tCj(t). 

By  the  definition  of  the  sampling  deviation  function,  the  dif¬ 
ference  between  Valsampied(ci)  and  controller  update  fXp,  Xc) 
at  time  t  must  be  contained  in  the  interval  u,',  (t).  Therefore, 
Valsampied(Ci)  is  contained  in  controller^pdatefXp,  Xc)  + 
ojjft).  The  continuous  approximation  at  time  t  is  equal  to 
controllerupdate.fXp,  Xc),  and  by  the  construction  of  the 
continuized  abstraction  from  the  continuous  approximation, 
the  inclusion  Valsampied(cj)  £  Valabstract (ci)  +  Wi{t)  holds. 

In  the  construction  of  the  continuous  abstraction,  each 
cyber  variable  c,  in  the  continuous  approximation  was  re¬ 
placed  by  a  +Wi{t).  Since,  as  shown  above,  Valsampied(cj)  £ 
Valabstract (cj)  +  the  derivatives  for  every  variable  in 


the  sampled  CPS  will  be  contained  in  the  derivatives  of 
continuized  abstraction.  In  particular,  the  physical  variable 
values  in  the  continuized  abstraction  also  contain  the  sampled 
CPS  physical  variable  values.  The  discrete  transitions  between 
the  two  systems  are  identical,  except  for  the  removal  of  the 
periodic  cyber-variable  updates  in  the  continuized  abstraction. 
Thus,  any  discrete  transition  (other  than  controller  updates, 
which  only  update  cyber  variables  and  for  which  we  already 
showed  containment)  taken  by  the  sampled  CPS  can  also 
be  taken  by  the  continuized  version.  Since  a  trajectory  is 
an  alternating  sequence  of  continuous  evolutions  and  discrete 
transitions,  and  the  initial  states  are  the  same,  by  induction  on 
the  length  of  a  trajectory,  the  values  of  the  physical  variables 
in  the  sampled  CPS  are  always  contained  in  the  values  of  the 
physical  variables  in  the  continuized  abstraction.  Therefore, 

reach("H)  4-  Xp  c  reach('Hc)  4-  Xp.  □ 

B.  Producing  Sampling  Deviation  Functions 

The  key  to  continuization  is  to  construct  sampling  deviation 
functions  that  provide  an  upper  and  lower  bound  on  the 
difference  of  each  cyber  variable  between  the  sampled  CPS 
and  the  controller  update  function.  One  way  to  compute  such 
a  function  is  by  looking  at  the  maximum  rate  of  change 
(bounded  by  a  Lipschitz  constant)  of  the  derivative  of  each 
cyber  variable  in  the  continuous  approximation.  This  process 
makes  use  of  standard  interval  arithmetic  multiplication,  [a,  6]* 
[c,d]  =  [min(a*c,a*d,6*c,&*d),max(a*c,a*d,&*c,&*d)]. 

Lemma  1  (Sampling  Deviation  using  Lipschitz  Constant). 
Given  interval  bounds,  K  =  [Kmm,  Kmax],  on  the  rate  of 
change  of  the  derivative  of  Ci  in  the  continuous  approximation, 
and  the  period  of  the  associated  strictly-periodic  ( Model  1 
from  Fig.  1 )  controller,  T,  a  sampling  deviation  function  is 

Ui  =  [-T,Q\*K. 

Proof.  The  sampling  deviation  function  needs  to  bound  the 
difference  of  the  value  of  the  variable  c,  in  a  sampled 
CPS  and  controller^pdatefX).  The  difference  between 
controller  update t  at  the  last  sample  time  (which  is  the 
current  value  of  the  cyber  variable  in  the  sampled  CPS),  and 
controller  update t  at  the  current  time  (which  is  its  value  in 
the  continuous  approximation)  is  at  most  a  product  of  the 
maximum  rate  of  change  K,  and  the  time  since  the  last  sample. 
The  difference  between  the  last  controller  update  and  the 
current  time  must  be  in  the  interval  [— T,  0],  since  it  is  a  strictly 
periodic  controller  with  period  T.  Assuming  the  first  sample 
occurs  at  time  0,  by  induction  on  the  number  of  samples,  this 
property  will  hold  for  every  sampling  period  and  therefore 
over  all  time.  □ 

In  this  case,  we  had  considered  a  strictly  periodic  controller, 
such  as  the  one  given  by  Model  1  in  Sec.  II-B.  To  compute 
the  function  for  a  nondeterministic  controller  such  as  Model 
3,  all  that  would  need  to  be  adjusted  is  the  time  of  the  last 
controller  update.  In  the  worst  case,  a  sample  will  occur  at  the 
start  of  one  period,  and  at  the  end  of  the  next  period.  In  that 
case,  the  maximum  time  between  updates  is  2  *  T,  so  using 
an  interval  of  [— 2  *  T,  0]  in  Lemma  1  would  be  adequate. 
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Continuization  of  a  Sine  Wave 


t 


Fig.  3:  The  main  idea  behind  the  proposed  continuization 
approach  is  that  a  nondeterministic  continuous  system  contains 
the  behaviors  of  a  periodically  sampled  system. 

To  provide  some  intuition  on  the  construction  of  sampling 
deviation  functions,  we  provide  an  simple  illustrative  example. 

Example  2  (Sine  Wave).  Consider  a  system  with  a  single 
cyber  variable  C\  where  the  controller_update  function  is  given 
by  sin(f),  and  t  is  a  clock  (physical  variable  with  t  =  1)  ticking 
from  0  to  7 r.  The  period  of  the  cyber-variable  is  T  =  0.2. 

The  rate  of  change  of  controller  update  (the  derivative)  is 
equal  to  cos(t),  and  the  bound  on  cos  in  [0, 7r]  is  K  =  [—1, 1]. 
Given  this  bound  and  the  period  of  T  =  0.2,  each  occurrence 
of  Ci  in  the  continuous  approximation  is  replaced  by  c,  + 
[—0.2,  0.2]  in  the  continuized  abstraction.  A  visual  depiction 
of  this  is  given  in  Fig.  3. 

One  nice  property  of  the  sampling  deviation  function  con¬ 
structed  by  Lemma  1  is  that  no  matter  how  large  the  bounds 
are  on  the  rate  of  change  of  the  controller_update  function,  the 
sampling  deviation  function  can  be  made  arbitrarily  small  by 
choosing  a  small  enough  controller  period  T.  This  is  because 
of  the  multiplication  in  the  sampling  deviation  function  by 
the  interval  [— T,  0].  Intuitively,  this  makes  sense,  since  the 
continuous  system  is  more  closely  approximated  as  we  sample 
and  actuate  at  a  higher  frequency.  This  is  in  contrast,  however, 
to  reachability  analysis  done  directly  on  the  sampled  CPS 
models,  where  smaller  periods  lead  to  more  discrete  transi¬ 
tions,  which  lead  to  more  error. 

The  width  of  the  interval  given  by  deviation  function  does 
affect  the  amount  of  overapproximation  in  the  constructed 
model,  and  therefore  it  is  desirable  to  have  this  function  be  as 
tight  as  possible.  One  way  to  improve  the  bound  on  the  rate  of 
change  of  the  cyber  variable  is  by  considering  smaller  domains 
(time  intervals).  For  example,  we  could  take  advantage  of  the 
time  dependence  of  each  sampling  deviation  function  to.,,  and 
define  corresponding  sampling  deviation  functions  within  local 
analysis  domains. 

Lemma  2  (Sampling  Deviation  in  Local  Analysis  Domains). 
For  a  cyber  variable  Ci  with  period  T,  given  a  sequence  of 
inten’al  bounds  on  the  rate  of  change  of  the  controller_update 
function,  K\ ,  K-2,  . . .,  Km,  and  an  associated  sequence  of 


Piecewise  Continuization  of  a  Sine  Wave 


Fig.  4:  The  continuization  approach  as  applied  to  four  local 
analysis  domains  has  an  overlap  of  one  period  length  between 
domains. 

increasing  and pointwise-intersecting  time  intervals  (which  we 
call  local  analysis  domains)  where  the  bounds  are  valid,  [to  = 
0,  fi],  [fi,  <2],  ■  ■  • ,  [fm_i,  tm],  a  sampling  deviation  function  up 
to  time  tm  can  be  computed  as: 

Ui(t)  =  [-T,  0]  *  [min ({Kfn  \  t  G  [iy-i,*?  +  T]}), 
ma x({I<™ax  |  f  G  [tj-utj  +  T]})]. 

Proof  Notice  the  time  intervals  have  the  controller  period 
T  added  to  the  upper  time  bound.  This  is  because  when  a 
new  time  interval  is  entered  in  a  trajectory,  the  sampled  CPS 
could  have  taken  the  most-recent  sample  in  either  the  current 
time  interval,  or  in  the  previous  one.  The  sampling  deviation 
function,  therefore,  must  account  for  both  possibilities  until  T 
time  has  elapsed  in  the  new  interval.  Other  than  this  caveat, 
the  proof  follows  that  of  Lemma  1,  except  that  the  analysis  is 
done  at  each  time  interval.  □ 

In  the  sine  wave  system  from  Example  2,  we  can  ap¬ 
ply  this  approach  in  four  analysis  domains  (time  inter¬ 
vals),  [0,  f  ],  [f ,  f],  [§,  ^f],  [^f,  1].  Solving  for  the  cos(f)  (the 
derivative  of  sin(f))  in  these  domains,  we  can  come  up  with 
the  associated  interval  bounds  on  i\  in  the  continuous  approx¬ 
imation,  Ki  =  [^,1  ],K2  =  [0  =  [-^,0  \,Ka  = 

[—1,  —  ^2].  Using  the  period  T  =  0.2,  we  then  obtain  the 
piecewise  continuization  of  the  system,  shown  in  Fig.  4. 

In  Fig.  4,  when  the  derivative  bounds  are  positive,  the 
difference  between  the  sampled  CPS  and  the  continuous 
approximation  is  negative,  which  is  why  an  interval  of  [— T,  0] 
was  used  to  bound  the  difference.  Also,  without  the  presence 
of  the  overlap  between  time  domains,  the  continuized  abstrac¬ 
tion  would  be  wrong  immediately  after  time  f .  In  this  case, 
the  new  domain  has  a  strictly  negative  derivative,  but  because 
the  sample  occurred  before  | ,  the  bound  from  the  previous 
domain  must  be  used. 

There  are  two  considerations  when  applying  continuization 
with  local  analysis  domains.  First,  the  result  is  only  valid  until 
the  maximum  time  of  the  last  analysis  domain.  If  this  time 
is  finite,  this  means  only  bounded-time  reachability  can  be 
computed.  Second,  there  is  a  trade  off  between  the  accuracy 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

37 


of  the  computation  and  the  number  of  domains  considered. 
Continuization  was  originally  used  to  eliminate  large  numbers 
of  discrete  transitions  in  a  sampled  CPS.  Using  local  analysis 
domains,  however,  brings  back  discrete  transitions,  although 
now  the  number  of  transitions  can  be  controlled  by  adjusting 
the  number  of  domains.  Using  too  many  domains  may  lead  to 
similar  problems  with  tool  performance  as  when  we  directly 
considered  a  sampled  CPS  model  for  reachability  analysis. 
We  could  solve  this  problem  by  having  sampling  deviation 
functions  that  vary  as  continuous  functions  of  time,  although 
the  way  to  create  these  is  less  clear,  and  left  as  possible  future 
work. 

IV.  Automation  in  Hyst 

Hyst  [8]  is  a  model  transformation  and  translation  tool 
for  hybrid  automaton  models.  Hyst  performs  both  model 
translation,  which  converts  between  formats  of  different  reach¬ 
ability  tools,  as  well  as  model  transformations,  which  serve  to 
improve  reachability  computation  results.  The  continuization 
approach  described  in  the  previous  section  has  been  imple¬ 
mented  as  a  model  transformation  pass  in  Hyst,  which  permits 
easy  application  of  the  developed  technique. 

A.  Transformation  Pass 

The  implemented  model  transformation  pass  performs  con¬ 
tinuization  starting  given  a  continuous  approximation  of  the 
system.  The  user  provides  (1)  a  target  model  file  describing 
the  hybrid  automaton,  (2)  the  controller  period,  T,  (3)  the 
name  of  the  cyber  variable  of  interest,  Ci,  (4)  a  sequence  of 
to  increasing  times  used  to  construct  local  analysis  domains, 
(5)  a  corresponding  sequence  of  to  bloating  terms,  which  will 
be  described  shortly,  and  (6)  the  name  of  the  time  variable 
(optional;  only  used  if  multiple  local  analysis  domains  are 
used  to  create  transitions  between  them). 

Given  these  inputs,  the  pass  first  simulates  the  continuous 
abstraction  from  the  center  of  the  initial  states,  in  order  to 
approximate  the  interval  bounds  on  the  rate  of  change  of 
the  derivative  of  Ci.  For  each  time  interval,  the  bound  during 
that  time  is  then  expanded  by  the  corresponding  user-provided 
bloating  term.  We  call  the  new  intervals  candidate  Lipschitz 
bounds  for  the  cyber  variable’s  derivative.  The  candidate 
Lipschitz  bounds  are  used  as  described  in  Lemma  2,  along 
with  the  time  domains,  in  order  to  produce  the  sampling 
deviation  function 

The  sampling  deviation  function  consists  of  piecewise  con¬ 
stant  intervals.  For  each  piece,  a  mode  is  created  in  the  output 
hybrid  automaton,  with  dynamics  equal  to  the  continuous 
approximation,  except  with  every  occurrence  of  Ci  replaced 
by  Ci  +0Ji(t).  Transitions  are  then  added  between  the  modes 
when  the  appropriate  amount  of  time  has  elapsed. 

The  bound  given  by  u>i  is  only  valid,  however,  if  the  candi¬ 
date  Lipschitz  bounds  are  actually  upper  and  lower  bounds  on 
the  derivative  of  the  cyber  variable.  This  can  happen  because 
the  bounds  are  constructed  from  a  single  simulation  using  the 
continuous  approximation,  whereas  the  reachable  set  of  states 
considers  all  initial  points  as  well  as  the  expanded  set  of  values 


for  the  cyber- variable  in  the  dynamics,  Ci  +  0Ji(t)  instead  of 
just  Ci.  To  check  if  the  bounds  are  respected,  invariants  and 
guards  are  added  to  the  output  hybrid  automaton  to  check  if 
the  derivative  exceeds  the  candidate  Lipschitz  bounds.  If  a 
violation  occurs,  a  transition  to  an  error  state  is  taken,  which 
is  added  as  a  forbidden  location  in  the  model.  In  this  way, 
performing  reachability  computation  will  not  only  give  the  set 
of  states  reachable  by  the  continuized  abstraction,  but  will  also 
check  that  the  candidate  Lipschitz  bounds  are  actual  bounds 
on  the  derivative  of  the  cyber  variable.  If  they  are  not,  the 
transition  to  the  error  state  will  be  detected  when  performing 
a  reachability  computation,  and  the  transformation  pass  can 
be  re-run  with  larger  bloating  terms,  which  will  increase  the 
size  of  the  candidate  Lipschitz  bounds. 

B.  Example 

We  apply  the  continuization  approach  in  Hyst  to  the  double¬ 
integrator  system  given  in  Example  1.  The  controller_update 
function  in  this  case  is  P  *  (1  —  x)  +  D  *  —v,  with  P  =  10 
and  D  =  3.  The  time  derivative  is  —10  *  x  —  3  *  v.  After 
substituting  in  the  derivatives  (x  =  v,  v  =  a),  the  derivative 
of  a  in  the  continuous  abstraction  is;  —10  *  v  —  3  *  a.  The 
initial  value  of  a  is  the  value  assigned  when  controller_update 
is  evaluated  at  the  initial  states,  a  :=  10*  (1  —  cc)  +  3* — v.  The 
hybrid  automaton  of  the  continuous  approximation  shown  in 
the  appendix,  in  Fig.  9. 

The  pass  implemented  in  Hyst  performs  a  simulation  of  the 
system  starting  from  the  center  of  the  initial  set  of  states,  in 
this  case,  at  x  =  0.05,  v  =  0,  a  =  9.5.  The  value  of  a  in 
the  simulation  is  observed  to  be  in  the  interval  [—28.64,  5.27]. 
This  interval  is  then  bloated  by  the  provided  bloating  term, 
for  which  we  consider  +1,  ±2  and  +4. 

When  running  reachability  with  a  bloating  term  of  1,  Flow* 
immediately  (at  time  0)  detects  that  the  constructed  error 
states  are  reachable,  which  means  that  the  candidate  Lipschitz 
bounds  do  not  contain  all  the  encountered  values  of  d.  Com¬ 
putationally,  we  can  show  this  to  be  the  case.  Initially,  x  = 
[0,  0.1],  which  means  the  initial  value  of  a  is  [9, 10].  The  initial 
value  of  a  is  —  10*u  —  3*a  =  [-30,-27].  The  interval  values 
of  d  in  the  simulation  were  [—28.64, 5.27],  which  bloated  by  1 
give  candidate  Lipschitz  bounds  of  [—29.64,6.27].  The  lower 
bound  of  the  derivative  of  the  cyber  variable  (—30)  is  initially 
outside  of  the  candidate  bounds,  which  was  detected  by  the 
transition  to  the  error  state. 

Using  a  bloating  term  of  2,  the  candidate  Lipschitz  bounds 
are  [—30.64,  7.27],  which  contain  the  above-computed  initial 
values  of  a.  When  performing  reachability,  however,  at  time 
0.04  an  error  state  is  reached  again.  At  this  time,  the  reachable 
set  contains  a  state  where  a  =  8.79  and  v  =  0.382.  In  this 
case,  the  derivative  d  =  — 10  *  u  —  3  *  a  +  =  —10*  0.382  — 

3  *  8.79  +  [—0.45,  0.11]  has  a  lower  value  of  —30.66,  which 
is  below  the  candidate  Lipschitz  bound  of  —30.64. 

When  the  larger  bloating  term  of  4  is  used,  the  candidate 
Lipschitz  bound  is  respected  by  the  reachable  set,  and  Flow* 
does  not  reach  the  out-of-bounds  error  states.  Thus,  the  reach 
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(a)  Continuized  System 


(b)  Two  Analysis  Domains 


Fig.  5:  The  response  for  the  continuized  periodically- 
controlled  double-integrator  system  from  Example  1  is  sig¬ 
nificantly  tighter  than  direct  analysis  (Fig.  2). 


Fig.  6:  The  impulse  response  for  the  washout  filter  design  of 
a  yaw  damper  demonstrates  the  spiral  mode  in  simulation. 


set  of  the  continuized  abstraction  is  a  validated  overapproxi¬ 
mation  of  the  reach  set  of  the  sampled  CPS. 

Recall,  however,  that  directly  computing  the  reach  set  of 
the  sampled  CPS,  as  shown  in  Fig.  2,  resulted  in  a  large 
exponential  blow  up  in  the  size  of  the  reachable  set  due  to 
accumulation  of  overapproximation  error.  Even  with  a  single 
analysis  domain,  the  reachable  set  is  significantly  smaller, 
as  shown  in  Fig.  5(a).  Using  multiple  analysis  domains,  the 
reachable  set  can  be  further  reduced.  The  hybrid  automaton  of 
the  continuized  system  with  two  local  analysis  domains  [0, 1.5] 
and  [1.5, 5]  is  shown  in  the  appendix  in  Fig.  10.  The  reach  set 
of  the  response  for  this  system  is  shown  in  Fig.  5(b).  Thus,  the 
continuization  method  developed  in  this  paper  enables  a  more 
precise  formal  analysis  of  this  system  using  hybrid  automaton 
reachability  tools. 

In  terms  of  overhead,  the  runtime  of  the  pass  itself  is  small, 
taking  about  100  ms.  The  reachability  computation  takes  0.9 
seconds  for  the  single-domain  case,  and  about  1.3  seconds  for 
the  two-domain  system,  which  is  significantly  faster  than  the 
12  minutes  needed  for  SpaceEx  to  produce  Fig.  2(b). 

V.  Case  Study 

In  this  section,  we  apply  the  technique  developed  from 
Sec.  Ill  in  order  to  perform  reachability  analysis  of  a  hybrid 
system  model  of  a  yaw-damper  for  a  747  aircraft. 

A.  System  and  Controller  Model 

The  model  and  controller  we  analyze  in  this  case  study 
are  taken  from  the  Control  Systems  Toolbox  case  studies  in 
Matlab  [13],  In  brief,  the  system  is  a  multiple-input  multiple- 
output  (MIMO)  system  that  uses  the  aileron  and  rudder  in 
order  to  reduce  oscillations  in  the  yaw  and  roll  angle. 

The  analysis  of  the  yaw  damper  is  done  on  the  sys¬ 
tem’s  aileron-to-bank  angle  impulse  response.  Three  different 
systems  are  considered:  (1)  the  original,  undamped  system, 
which  experiences  oscillations  upon  an  impulse  input,  (2) 
the  system  with  proportional  compensator,  which  eliminates 
the  oscillations  but  also  over-stabilizes  the  spiral  mode  (a 
desired  characteristic  for  the  control),  and  (3)  the  system  with 
a  washout  filter,  which  eliminates  the  oscillations  but  keeps 
the  spiral  mode. 

We  use  this  case  study  to  evaluate  the  developed  con¬ 
tinuization  technique  so  as  to  evaluate  properties  about  the 
response  of  the  final  (washout  filter)  system.  There  are  four 


physical  variables  in  this  system,  sideslip  angle  (x-\ ).  yaw  rate 
( X2 ),  roll  rate  (X3),  and  bank  angle  (xf),  represented  by  the 
column  vector  x.  The  two  inputs  u,  are  the  rudder  (ui  )  and 
aileron  (0.2).  The  outputs  are  the  yaw  rate  and  bank  angle. 
The  dynamics  for  the  physical  system  are  the  standard  linear 
time-invariant  dynamics,  x  =  Ax  +  Bu  (the  A  and  B  matrices 
are  provided  in  the  in  Sec.  B  of  the  appendix). 

This  physical  system  is  put  into  a  feedback  loop  with  a 
washout  filter.  The  washout  filter  has  a  single  variable,  w, 
with  dynamics  w  =  X2  —  0.2  *  w.  The  washout  filter  variable 
is  combined  with  the  yaw  to  produce  an  effect  on  the  rudder 
input.  That  is,  the  washout  filter  adds  to  v,\  the  value  2.34  * 
(X2  —  0.2  *  w). 

A  simulation  of  the  aileron-to-bank  angle  impulse  response 
from  this  system,  with  and  without  the  washout  filter,  is  given 
in  Fig.  6.  In  particular,  the  two  control  properties  of  interest 
are  a  lack  of  oscillations  (quick  settling  time),  and  the  presence 
of  the  spiral  mode.  The  spiral  mode  is  a  desirable  flight 
characteristic  demonstrated  by  the  apparent2  steady-state  offset 
in  the  rudder-to-bank  angle  impulse  response. 

A  property  to  check  is  that  the  aileron  to  bank  angle  impulse 
response  remains  around  the  simulated  value  of  0.08,  between 
20  and  40  seconds,  and  thus  maintains  the  spiral  mode  without 
significant  oscillation.  We  consider  a  controller  running  at  20 
Hz  (T  =  0.05),  using  the  implementation  that  samples  and 
actuates  when  the  real-time  scheduled  controller  runs  (Model 
3  from  Sec.  II-B). 

B.  Reachability  Analysis 

Neither  SpaceEx  nor  Flow*  can  effectively  compute  reach¬ 
ability  on  the  periodically-actuated  system  model  (Fig.  11  in 
the  appendix).  The  reachable  set  of  states  explodes  almost 
immediately,  and  neither  tool  can  compute  accurate  time- 
bounded  reachability  for  the  required  40  seconds. 

We  apply  the  continuization  approach  developed  in  this 
paper  by  using  the  Hyst  transformation  pass  on  the  continuous 
approximation  of  the  model.  First,  we  apply  the  technique  over 
the  whole  time  range.  Initially,  we  try  a  small  bloating  term, 
and  increase  it  until  error  states  are  no  longer  reachable  during 
analysis.  For  the  period  parameter  given  to  the  pass,  we  use 
twice  the  control  period,  as  this  is  needed  to  account  for  the 

2The  steady  state  is  actually  zero,  but  the  convergence  is  very  slow  over 
hundreds  of  seconds. 
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(a)  Reachability  in  Flow*  of  (b)  Reachability  with  Local 
the  Continuized  Model  Domains  and  Halving  Period 

Fig.  7:  Flow*  can  successfully  compute  reachability  on  the 
continuized  model.  When  a  smaller  period  and  local  analysis 
domain  is  used,  the  result  is  tighter. 

maximum  delay  in  sampling  in  Model  3,  as  discussed  earlier 
in  Sec.  II-B.  Flow*  successfully  computes  reachability  for  the 
model,  and  confirms  that  the  final  bloating  term  (0.0007)  was 
sufficiently  large.  The  output  plot  is  shown  in  Fig.  7(a). 

Although  the  computation  completes,  which  is  an  improve¬ 
ment  over  the  direct  computation,  the  set  of  states  appears  to 
be  diverging  slowly.  The  reachability  result  can  be  improved 
by  using  local  analysis  domains,  or  by  reducing  the  controller 
period.  To  demonstrate  this,  we  halve  the  controller  period,  and 
use  two  analysis  domains.  For  time  [0, 8]  we  use  a  bloating 
term  of  0.0004,  and  for  time  [8,40]  we  use  0.0003.  Hyst 
creates  the  associated  model  file  for  Flow*,  which  we  then  use 
to  compute  reachability.  Flow*,  in  about  5  seconds,  confirms 
that  the  candidate  domains  are  sufficient,  and  the  resultant 
reachability  plot  is  tighter  than  the  previous  one,  as  shown  in 
Fig.  7(b).  Furthermore,  the  spiral  mode  can  be  observed  from 
the  reachable  set  plot,  along  with  the  absence  of  oscillations 
in  the  time  range  [20,40]. 

VI.  Related  Work 

In  this  paper,  we  have  focused  on  controller  analysis  using 
hybrid  automata  reachability  tools,  although  there  are  existing 
methods  in  control  theory  to  design  and  analyze  controllers. 
The  design  of  a  controller  for  a  continuous-time  system  often 
occurs  in  continuous-time,  and  the  controller  is  subsequently 
discretized3  to  be  implemented  in  a  software  controller  that 
operates  periodically. 

Continuous-Time  Controller  Design:  There  are  many 
methods  for  control  design  in  continuous-time.  For  example,  a 
common  strategy  for  linear  time-invariant  (LTI)  systems  is  to 
design  a  stabilizing  linear  state-feedback  controller  of  the  form 
u  =  Kx  for  a  vector  K  [16],  Assuming  the  system  is  both 
controllable  and  observable,  the  strategy  yields  a  new  closed- 
loop  system:  x  =  Ax  +  Bu  for  u  =  Kx.  After  substituting 
this  gives  x  =  Ax  +  B(I\x)  and  then  x  =  {A  +  BK)x.  This 
strategy  is  also  known  as  pole  placement  [16].  Finding  the 
vector  K  such  that  (A  +  BK )  is  exponentially  stable  can  be 
formulated  in  a  variety  of  ways,  such  as  by  solving  a  linear  ma¬ 
trix  inequality  (LMI)  [17].  Linear  quadratic  regulator  (LQR) 

3 In  this  paper,  we  only  focus  on  the  conversion  from  continuous-time  to 
discrete-time,  and  do  not  consider  full  digitization  [14],  [15],  for  example, 
the  conversion  from  continuous-time  and  continuous-state  to  discrete-time  and 
discrete-state  through  quantization. 


design  is  another  linear  system  design  technique  that  also 
incorporates  a  cost  function  to  yield  an  optimal  controller  [18]. 
LQR  is  used  within  the  Linear  Quadratic  Gaussian  (LQG) 
problem  that  robustly  tolerates  Gaussian  additive  noise  inputs 
from  disturbances.  Other  control  design  methods  for  linear 
systems  are  performed  in  the  frequency  domain,  where  pole 
and  zero  placement  may  also  be  performed  to  ensure  stability 
and  analyze  performance  criteria  such  as  gain  margins,  phase 
margins,  and  use  graphical  tools  like  Nyquist  diagrams  and 
Bode  plots.  Design  of  controllers  for  nonlinear  systems  is 
challenging,  but  many  approaches  exist,  such  as  linearizing 
and  using  gain-scheduled  linear  controllers,  backstepping, 
feedback  linearization,  and  many  others  [19]. 

Discretization  of  Continuous  Controllers:  Discretization 
typically  consists  of  several  steps.  First,  a  sampling  period 
must  be  selected  at  which  measurements  of  the  physical 
system  are  taken  and  made  available  to  the  software  controller. 
Second,  a  control  period  must  be  selected  to  specify  the  rate 
at  which  control  decisions  are  produced  by  the  software  con¬ 
troller  and  sent  to  actuators  to  influence  the  plant.  Typically, 
these  periods  are  selected  in  accordance  with  the  speeds  of  the 
dynamics,  and  a  common  rule  of  thumb  is  to  use  the  Nyquist 
frequency  of  the  physical  process  to  determine  the  minimum 
sampling  period.  The  Nyquist  frequency  is  twice  the  highest 
waveform  frequency. 

Given  these  periods,  a  discrete-time  version  of  the  plant 
can  be  constructed  (using  the  sampling  period)  and  a  discrete¬ 
time  version  of  the  controller  can  be  constructed  (using  the 
control  period).  Both  discretizations  are  needed,  as  from 
the  perspective  of  the  controller,  it  will  only  receive  state 
measurements  of  the  plant  at  the  points  in  time  specified  by 
the  sampling  period. 

Discrete  Controllers  with  Continuous  Plants:  While  from 
the  perspective  of  the  software  controller,  the  changes  to  the 
plant  occur  discretely,  in  reality,  the  plant  evolves  continuously 
according  to  differential  equations.  Controller  performance 
with  such  constraints  has  been  extensively  investigated,  and 
tools  like  JitterBug  and  TrueTime  can  characterize  controller 
performance  with  real-time  constraints  and  delays  [20].  More 
recent  works  aid  in  synthesizing  embedded  software  from 
hybrid  systems  models  [21],  Giotto  aids  in  this  process  of 
moving  from  control  models  to  embedded  real-time  code  [22]. 

Reachability:  The  elimination  of  large  numbers  of  dis¬ 
crete  transitions  in  hybrid  automata  was  previously  accom¬ 
plished  by  continuization  [7].  The  earlier  work  was  used  to  an¬ 
alyze  properties  about  fast-switching  electronic  circuits.  This 
work,  in  contrast,  applied  continuization  to  enable  the  analysis 
of  fast-switching  hybrid  automata  resulting  from  the  periodic 
interactions  with  the  real-time  scheduler.  We  also  considered 
using  local  analysis  domains  to  construct  the  nondeterministic 
term,  which  was  shown  to  increase  the  accuracy  of  the  model. 

Periodically  Controller  Hybrid  Automata  (PCHA)  is  one 
formalism  for  periodically-controlled  embedded  systems  [23]. 
Automated  analysis  of  PCHAs  is  possible  only  if  the  vector 
fields  are  polynomial,  whereas,  using  the  developed  Hyst 
pass,  continuization  can  be  automatically  applied  to  a  broader 
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class  of  systems.  Combinations  of  reachability  tools  and  SMT 
solvers  have  been  used  to  model  both  physical-world  dynamics 
and  software  behavior  [24].  A  limitation  of  this  approach  is 
that  cyber-variables  are  represented  with  intervals,  and  that 
only  strictly-periodic  systems  can  be  analyzed  (Model  1  from 
Sec.  II-B). 

VII.  Conclusion 

Analysis  of  large  CPS  using  formal  hybrid  systems  anal¬ 
ysis  techniques  remains  difficult.  A  challenge  problem  was 
recently  proposed  to  the  research  community  by  Toyota  on 
the  verification  of  a  powertrain  control  system  [25].  Although 
initial  progress  has  been  made  on  simplified  versions  of  the 
system  [26],  the  full  benchmark  model  presents  four  main 
challenges  for  verification  tools:  (1)  controllers  that  periodi¬ 
cally  actuate  the  plant,  (2)  lookup  tables  to  describe  the  system 
dynamics,  (3)  the  presence  of  time  delays  in  the  model,  and 
(4)  large  system  scale. 

In  this  paper,  we  addressed  the  first  of  these  issues,  by  using 
continuization  in  order  to  soundly  abstract  the  periodically- 
controlled  dynamics.  This  permits  initial  analysis  of  these 
systems  using  reachability  tools  for  hybrid  automata.  Without 
our  approach,  existing  tools  produce  exponentially  divergent 
reach  sets  on  these  models,  and  often  fail  before  reaching  the 
desired  time  bound.  Since  the  accuracy  of  analysis  depends  on 
the  tightness  of  the  difference  between  the  discrete  system  and 
continuized  abstraction,  a  possible  future  improvement  would 
be  to  compute  these  bounds  in  local  domains  based  on  the 
system  state,  in  addition  to  time  as  proposed  in  this  paper. 
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Guard:  c  =  0.05? 
a  :=  10  *  (1  —  x)  +  3  *  (— v) 
c:=  0 


Fig.  8:  Hybrid  automaton  model  for  sampled  CPS  of  the 
double -integrator  system  in  Example  1. 


Fig.  9:  Hybrid  automaton  model  for  continuous  approximation 
of  the  double-integrator  system  in  Example  1. 


Fig.  10:  Hybrid  automaton  model  for  continuized  abstraction 
with  two  analysis  domains  (with  error  modes  and  transitions 
omitted)  of  the  double-integrator  system  in  Example  1. 


0.0  0.5  1.0  1.5  2.0  2.5 

(a)  SpaceEx 

Fig.  11:  Neither  SpaceEx  (left)  nor  Flow*  (right)  can  directly 
compute  reachability  accurately  on  the  yaw-damper  model. 


Fig.  12:  The  continuous  approximation  of  the  yaw-damper 
system  demonstrates  the  spiral  mode. 

In  Fig.  10,  the  error  modes  and  transitions  were  not  drawn. 
The  guard  conditions  to  enter  an  error  mode  in  the  first  domain 
are  —  10*u  —  3*a+0.139  >  9.27  or  —  10*u  —  3*aH — 0.490  < 
—32.65.  In  the  second  domain,  the  guard  conditions  are  10  * 
u-3*a  +  0.109>  7.24  or  -10*u-3*a  +  -0.075  <  -4.97. 

B.  Yaw-Damper  Example 

The  dynamics  of  the  yaw-damper  system  from  Sec.  V  are 
standard  linear  time-invariant  dynamics,  x  =  Ax  +  Bu ,  with: 


(b)  Flow* 


Appendix 

A.  Double-Integrator  Example 

The  hybrid  automata  for  the  double-integrator  system  (Ex¬ 
ample  1)  are  shown  in  Figs.  8,  9,  and  10.  In  the  continuous 
approximation  and  the  continuized  abstraction,  the  initial  value 
of  a  is  taken  to  be  the  value  when  controller  update  is 
evaluated  at  the  initial  states,  a  :=  10  *  (1  —  x)  +  3  *  —v. 

The  continuized  abstraction  shown  in  Fig.  10  is  constructed 
from  two  time  domains,  [0, 1.5]  and  [1.5,5],  using  a  bloating 
term  of  4  for  each  of  the  domains.  The  ranges  of  a  in  simula¬ 
tion  for  the  two  domains  are  [—28.65,  5.27]  and  [—0.97,  3.24], 
which  give  interval  bounds  of  K\  =  [—32.65, 9.27],  and 
K?  =  [—4.97,  7.24].  With  a  period  of  T  =  0.005,  this  gives  in¬ 
terval  values  for  u>  of  [—0.046, 0.163]  and  [—0.036, 0.025].  The 
derivative  a  uses  a  value  of  —3  multiplied  by  these  intervals 
due  to  the  substitution  of  a  by  a  +  w  (since  a  is  multiplied  by 
—3  in  the  derivative).  The  derivative  could  have  equivalently 
been  written  as  a  =  —10  *  v  —  3  *  (a  +  [—0.036,  0.025]). 


A  = 


-0.0558  -.9968 

0.598  -0.115 

-3.05  0.388 

0  0.0805 


0.0802  0.0415 

-0.0318  0 

-0.4650  0 

1  0 


0.00729  0 

-0.475  0.00775 

0.153  0.143 

0  0 


Neither  SpaceEx  nor  Flow*  can  compute  reachability  on  the 
periodically-actuated  system.  The  reachability  plots  produced 
by  the  reachability  tools  on  the  real-time  actuated  model 
(Model  3)  are  given  in  Fig.  11. 

The  continuous  approximation  of  the  system  demonstrates 
the  spiral  mode  and  is  close  to  the  reach  set  for  the 
periodically-actuated  washout  filter  system.  The  plot  for  the 
continuous  approximation  is  shown  in  Fig.  12.  This  is  the 
system  that  is  used  as  input  to  the  Hyst  continuization  pass. 
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ABSTRACT 

Hybridization  methods  enable  the  analysis  of  hybrid  au¬ 
tomata  with  complex,  nonlinear  dynamics  through  a  sound 
abstraction  process.  Complex  dynamics  are  converted  to 
simpler  ones  with  added  noise,  and  then  analysis  is  done  us¬ 
ing  a  reachability  method  for  the  simpler  dynamics.  Several 
such  recent  approaches  advocate  that  only  “dynamic”  hy¬ 
bridization  techniques — i.e.,  those  where  the  dynamics  are 
abstracted  on-the-fly  during  a  reachability  computation — 
are  effective.  In  this  paper,  we  demonstrate  this  is  not  the 
case,  and  create  static  hybridization  methods  that  are  more 
scalable  than  earlier  approaches. 

The  main  insight  in  our  approach  is  that  quick,  numeric 
simulations  can  be  used  to  guide  the  process,  eliminating 
the  need  for  an  exponential  number  of  hybridization  do¬ 
mains.  Transitions  between  domains  are  generally  time- 
triggered,  avoiding  accumulated  error  from  geometric  inter¬ 
sections.  We  enhance  our  static  technique  by  combining 
time-triggered  transitions  with  occasional  space-triggered 
transitions,  and  demonstrate  the  benefits  of  the  combined 
approach  in  what  we  call  mixed-triggered  hybridization.  Fi¬ 
nally,  error  modes  are  inserted  to  confirm  that  the  reachable 
states  stay  within  the  hybridized  regions. 

The  developed  techniques  can  scale  to  higher  dimensions 
than  previous  static  approaches,  while  enabling  the  paral¬ 
lelization  of  the  main  performance  bottleneck  for  many  dy¬ 
namic  hybridization  approaches:  the  nonlinear  optimization 
required  for  sound  dynamics  abstraction.  We  implement  our 
method  as  a  model  transformation  pass  in  the  HYST  tool, 
and  perform  reachability  analysis  and  evaluation  using  an 
unmodified  version  of  SpaceEx  on  nonlinear  models  with  up 
to  six  dimensions. 
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1.  INTRODUCTION 

A  hybrid  automaton  [7]  is  an  expressive  mathematical 
model  useful  for  describing  complex  dynamic  processes  in¬ 
volving  both  continuous  and  discrete  states  and  their  evolu¬ 
tion.  Efficient  algorithms  and  analysis  tools  for  linear  and 
affine  systems  have  recently  emerged  [24] .  However,  the  be¬ 
haviour  of  many  real-world  systems  can  only  be  modeled 
with  nonlinear  differential  equations. 

Hybridization  methods  attempt  to  address  this  issue,  en¬ 
abling  the  application  of  existing  algorithms  for  simpler  dy¬ 
namics  (such  as  constant  or  affine  dynamics)  on  the  analy¬ 
sis  of  hybrid  automata  with  nonlinear  differential  equations. 
Alternative  recent  approaches  for  analyzing  nonlinear  sys¬ 
tems  include  simulation-based  verification  [22]  or  using  effi¬ 
cient  representations  such  as  Taylor  models  [17].  Most  hy¬ 
bridization  methods  work  by  dividing  the  state  space  into  a 
set  of  domains.  In  each  domain,  the  nonlinear  dynamics  are 
then  converted  to  simpler  ones  with  added  noise  to  account 
for  the  abstraction  error  within  the  domain.  Hybridization 
is  also  known  as  conservative  approximation  [8],  which  il¬ 
lustrates  that  it  is  a  sound  (or  conservative)  abstraction. 
Hybridization  has  been  used  to  verify  properties  for  several 
types  of  systems,  from  analog/mixed-signal  circuits  [19]  to 
autonomous  satellite  maneuvers  in  space  [14,31]. 

We  classify  existing  hybridization  approaches  along  two 
axes  as  shown  in  Table  1:  static  versus  dynamic,  and  space- 
triggered  versus  time-triggered.  Static  hybridization  ap¬ 
proaches  use  a  fixed  partitioning,  and  can  make  use  unmod¬ 
ified,  off-the-shelf  analysis  tools.  In  contrast,  dynamic  meth¬ 
ods  exploit  runtime  information  to  perform  hybridization, 
and  therefore  must  be  tightly  integrated  within  an  analysis 
tool.  On  the  other  axis,  space-triggered  techniques  perform 
geometric  intersections  along  hybridization  domain  bound¬ 
aries.  Time-triggered  hybridization,  on  the  other  hand, 
avoids  this  operation  by  creating  a  series  of  overlapping 
domains,  and  switches  between  them  at  specific  points  in 
time. 

Based  on  this  classification,  a  gap  exists  in  existing  re¬ 
search:  no  methods  exist  that  perform  static,  time-triggered 
hybridization.  The  main  contribution  of  this  paper  is  the 
investigation  of  this  category,  and  demonstrating  that  such 
methods  can  overcome  some  of  the  drawbacks  of  existing 
hybridization  methods.  Notably,  the  new  hybridization 
methods  are  more  scalable  than  existing  space-triggered 
approaches.  Furthermore,  the  expensive  dynamics  abstrac¬ 
tion  step,  which  is  generally  a  global  optimization  problem, 
is  easily  parallelizable,  which  is  not  the  case  in  dynamic 
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Space- 

Triggered 

Time- 

Triggered 

Mixed- 

Triggered 

Static 

[8, 10,29,31] 

this  paper 

this  paper 

Dynamic 

[8,9] 

[1-3,5,20,28] 

none 

Table  1:  Breakdown  of  hybridization  approaches  into  static 
versus  dynamic,  and  space-triggered  versus  time-triggered, 
as  well  as  combinations  thereof  (mixed-triggered) . 


approaches.  We  further  enhance  our  static  technique  by 
combining  time-triggered  transitions  with  occasional  space- 
triggered  transitions,  and  demonstrate  the  benefits  of  the 
combined  approach  in  what  we  call  mixed-triggered  hy¬ 
bridization. 

The  static  mixed-triggered  hybridization  approach  works 
by  hybridizing  only  a  part  of  the  state  space.  We  use  quick 
numeric  simulations  to  guide  the  partitioning  process.  In 
this  way,  we  mitigate  the  problem  of  exponential  growth  in 
the  number  of  partitions.  In  addition,  we  generally  use  time- 
triggered  guards  in  the  transitions  between  partitions.  This 
prevents  costly  geometric  intersection  computations  which 
typically  add  overapproximation  error  to  the  result.  We  en¬ 
sure  the  soundness  of  the  constructed  abstraction  by  adding 
error  modes  to  guarantee  that  the  computed  reachable  states 
remain  within  the  hybridized  region  (which  is  constructed 
from  simulations  that  may  be  imprecise). 

We  implement  the  hybridization  method  described  in  this 
paper  as  a  model  transformation  pass  in  the  Hyst  source-to- 
source  translation  tool.  Since  it  is  a  static  approach,  we  can 
use  unmodified  reachability  tools  on  the  hybridized  models. 
We  create  affine  abstractions  of  nonlinear  dynamics,  and  use 
to  perform  reachability  analysis. 

Contributions  and  Paper  Organization.  The  main  con¬ 
tribution  of  this  paper  is  the  development  of  the  first  static 
time-triggered  and  mixed-triggered  hybridization  methods. 
Of  critical  importance  in  the  proposed  approaches  is  the 
choice  of  hybridization  parameters,  and  a  second  contribu¬ 
tion  is  an  algorithm  which  uses  simulations  to  generate  these 
values.  This  algorithm  is  implemented  in  the  Hyst  [12] 
model  transformation  tool,  which  allows  it  to  quickly  be 
applied  to  new  systems  and  with  new  simulation  param¬ 
eters.  Finally,  we  validate  our  claims  that  the  method  is 
more  scalable  than  existing  static  approaches  by  evaluat¬ 
ing  it  on  nonlinear  models,  including  a  six-dimensional  wa¬ 
ter  tank  model,  and  then  using  an  unmodified  version  of 
SpaceEx  [13,15,24],  which  does  not  natively  support  non¬ 
linear  dynamics,  to  compute  the  set  of  reachable  states. 

This  paper  first  reviews  and  classifies  existing  hybridiza¬ 
tion  methods  in  Section  2.  Section  3  then  presents  math¬ 
ematical  background  and  formalisms,  which  are  used  in 
Section  4  to  give  formal  descriptions  and  correctness  ar¬ 
guments  for  several  hybrid  automaton  transformations.  A 
simulation-based  algorithm  to  create  the  hybridization  pa¬ 
rameters  used  by  the  transformations  is  described  next  in 
Section  5.  Section  6  discusses  the  implementation  in  Hyst 
and  experimental  reachability  results  in  SpaceEx,  followed 
by  a  conclusion  in  Section  7. 

2.  HYBRIDIZATION  METHODS 

In  this  section,  we  discuss  and  classify  previous  research 
on  hybridization.  Hybridiziation  is  the  process  of  using  sim¬ 
ple  dynamics  with  noise  to  create  an  abstraction  of  a  system 
with  more  complicated,  usually  nonlinear,  dynamics.  This  is 


done  to  enable  the  analysis  of  systems  with  the  more  compli¬ 
cated  dynamics  by  methods  which  work  exclusively  on  the 
simpler  ones. 

This  process  is  typically  targeted  for  flow-pipe  construc¬ 
tion  methods,  where  the  set  of  reachable  states  is  iteratively 
computed  or  overapproximated  at  monotonically  increasing 
instances  in  time,  starting  from  an  initial  set  of  states.  Com¬ 
putational  approaches  maintain  some  representation  of  the 
set  of  states  at  each  time  instances,  which  we  informally 
refer  to  as  the  currently-tracked  set  of  states. 

Static  Space-Triggered  Hybridization.  Early  hybridiza¬ 
tion  methods  were  both  static  and  space-triggered  [29].  In 
these  approaches,  the  state  space  is  partitioned  using  a 
(typically  uniform)  grid  or  mesh,  and  transitions  are  added 
along  the  partition  boundaries,  resulting  in  state-dependent 
switching.  The  advantage  of  this  approach  is  that  exist¬ 
ing  termination  checking  techniques  can  be  used,  which  is 
particularly  useful  in  the  case  of  periodic  systems  where 
linearizing  a  bounded  subset  of  the  state-space  is  reason¬ 
able  [31]. 

There  are,  however,  three  main  drawbacks.  First,  static 
mesh  construction  is  traditionally  done  without  knowledge 
of  the  reachable  states.  Therefore,  it  requires  computing  the 
mesh  over  the  entire  state  space  (or  bounded  subset  thereof) , 
which  scales  exponentially  with  the  number  of  continuous 
dimensions  in  the  system.  Second,  the  geometric  intersec¬ 
tions  required  by  space-triggered  approaches  may  introduce 
error  during  reachability  computation  [4,  17].  This  is  be¬ 
cause  such  intersections  can  require  tools  to  convert  from 
precise  internal  representations  such  as  zonotopes  [25] ,  sup¬ 
port  functions  [27],  or  Taylor  models  [17],  to  simpler  repre¬ 
sentations  where  intersection  operations  can  be  computed, 
such  as  polytopes  [6].  After  intersection,  the  simpler  rep¬ 
resentation  is  then  converted  back  to  the  internal  represen¬ 
tation  for  subsequent  computation  [26].  These  conversions 
can  result  in  overapproximations  of  the  original  currently- 
tracked  set  of  states,  adding  error  each  time  they  are  per¬ 
formed.  Since  hybridization  can  be  done  more  accurately 
when  domains  are  small,  many  intersection  operations  may 
be  necessary  and  this  can  quickly  lead  to  error  explosion, 
as  well  as  an  explosion  in  the  number  of  modes  of  the  hy¬ 
brid  automaton.  Third,  the  currently-tracked  set  of  reach¬ 
able  states  may  leave  a  hybridization  domain  along  multiple 
facets,  requiring  splitting  and,  later,  possibly  remerging  the 
set  of  reachable  states,  which  can  be  both  computationally 
expensive  and  inaccurate  [20]. 

Dynamic  Space-Triggered  Hybridization.  In  order  to 
help  increase  scalability,  methods  were  developed  that  per¬ 
form  hybridization  during  reachability  analysis  [8].  This 
results  in  dynamic  methods  where  the  domain  construction 
and  the  abstraction  process  is  performed  on-the-fly  and  only 
on  states  that  are  reachable  [9].  Although  dynamic  space- 
triggered  methods  scale  better  into  higher  dimensions,  they 
still  suffer  from  the  other  two  problems  mentioned  above: 
error  accumulation  due  to  many  geometric  intersections, 
and  the  splitting  of  the  currently-tracked  set  of  states  along 
multiple  facets. 

Dynamic  Time-Triggered  Hybridization.  To  address  the 
other  two  drawbacks,  dynamic  time-triggered  approaches 
were  developed  [5,20,28].  These  methods  avoid  geometric 
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intersections  by  choosing  hybridization  domains  around  the 
currently-tracked  set  of  states.  As  time  is  advanced,  the  hy¬ 
bridization  domains  are  updated  to  be  near  the  new  position 
of  the  currently-tracked  set  of  states,  without  requiring  an 
intersection  operation.  This  can  be  done  at  each  step  [28], 
or  whenever  the  currently-tracked  set  of  states  leaves  the  hy¬ 
bridization  domain  [20] .  This  can  be  viewed  as  the  mode  of 
the  abstract  hybrid  automaton  changing  at  specific  instances 
in  time  to  a  mode  with  new  dynamics,  which  corresponds 
to  a  time-triggered  transition. 

Although  dynamic  time-triggered  methods  perform  well, 
they  also  suffer  from  certain  drawbacks.  The  most  impor¬ 
tant  drawback  is  that,  in  the  earlier  static  approaches,  per¬ 
forming  the  dynamics  abstraction  step  was  an  embarrass¬ 
ingly  parallel  problem,  so  parallelism  could  be  leveraged 
to  reduce  total  runtime  (or  equivalently,  increase  precision 
for  a  fixed  runtime).  In  dynamic  methods,  the  bounds  of 
each  new  abstraction  domain  depend  on  the  set  of  reachable 
states  in  the  previous  domain,  forcing  this  expensive  step  to 
be  performed  serially.  For  example,  abstracting  nonlinear 
dynamics  using  polynomial  differential  inclusions  can  yield 
an  accurate  hybridization,  but  it  requires  bounding  the  La¬ 
grange  remainder  of  the  dynamics’  Taylor  expansion  [1] .  In 
previous  work,  this  step  was  reported  to  take  1121  out  of 
1180  seconds  on  a  nine-dimensional  biological  aging  model 
(about  95%  of  the  runtime),  and  1155  out  of  1296  seconds 
on  hybrid  variant  of  the  same  model  (about  89%),  although 
it  was  mentioned  that  some  implementation  optimizations 
were  possible  [1].  Some  parallelization  of  reachability  com¬ 
putation  was  considered  to  enable  online  reachability  of  car 
manoeuvres  [2,3].  However,  the  crucial  step  of  dynamics  ab¬ 
straction  (computing  the  linearization  errors)  was  still  per¬ 
formed  serially  because  the  overapproximation  of  the  La¬ 
grange  remainders  of  the  Taylor  expansions  of  the  dynamics 
at  each  step  was  based  on  the  Lagrange  remainders  at  the 
previous  step.  This  serial  step  dominated  the  reported  run¬ 
time  of  the  technique. 

A  second  drawback  of  time-triggered  approaches  is  that,  if 
the  currently-tracked  set  of  states  becomes  large  (which  can 
be  a  property  of  the  system  regardless  of  the  method  used), 
the  domains  over  which  dynamics  abstraction  is  performed 
also  become  large.  This,  in  turn,  increases  the  dynamics 
approximation  error  that  must  be  added  to  the  simpler  dy¬ 
namics  to  result  in  a  sound  abstraction,  increasing  error  in 
the  overapproximation  of  the  set  of  reachable  states.  This 
can  be  overcome  by  splitting  the  set  of  reachable  states  [21], 
although  this  may  yield  an  exponential  number  of  sets  that 
need  to  be  tracked,  and  possibly  redundant  computation. 
This  problem  can  be  partially  mitigated  through  extra  track¬ 
ing  to  perform  cancellation  of  redundant  sets  of  reachable 
states,  which  requires  (expensive  and  error-introducing)  in¬ 
tersection  operations  on  the  internal  representations  [5]. 
Space-triggered  approaches  do  not  suffer  from  this  prob¬ 
lem.  In  fact,  introducing  occasional  artificial  space-triggered 
transitions  can  serve  to  reduce  the  size  and  complexity  of 
the  currently-tracked  set  of  reachable  states  [11]. 

Novel  Hybridization  Approaches.  A  classification  of  ex¬ 
isting  hybridization  research  is  shown  in  Table  1.  A  research 
gap  is  noticeable  in  the  static  time-triggered  category.  This 
paper  attempts  to  fill  this  gap  by  developing,  to  the  best 
of  the  authors’  knowledge,  the  first  static  time-triggered  hy¬ 
bridization  method.  The  approach  is  static,  and  therefore 


can  perform  the  bottleneck  step  of  dynamics  abstraction  in 
a  parallel  fashion.  Since  the  approach  is  time-triggered,  it 
can  scale  to  larger  numbers  of  dimensions  while  avoiding 
the  accumulation  of  intersection  error.  Additionally,  as  the 
method  is  static  and  modifies  the  model  directly,  it  can  work 
with  unmodified  reachability  tools,  yielding  immediate  ben¬ 
efit  of  its  application  using  the  latest  reachability  methods. 

There  are  also  no  fundamental  reasons  why  a  method 
could  not  use  both  time-triggered  and  space-triggered  tran¬ 
sitions  during  analysis.  We  develop  such  a  mixed-trig gered 
hybridization  approach,  which  generally  uses  time-triggered 
transitions,  but  occasionally  performs  a  state-triggered  tran¬ 
sition  to  attempt  to  reduce  the  size  and  complexity  of  the 
currently-tracked  set  of  states.  In  our  review  of  existing 
research,  no  such  approaches  currently  exist. 

Other  Hybridization  Factors.  Research  in  hybridization 
also  explores  other  aspects  that  are  important,  but  less  criti¬ 
cal  to  the  methods  developed  in  this  paper.  One  choice  when 
performing  hybridization  is  the  shape  of  space-triggered  do¬ 
mains.  Rectangular  domains  are  simple  to  reason  about, 
although  manual  region  selection  [29],  simplexes  [9,21,31], 
and  nonuniform  meshes  [8,10,31]  have  been  considered.  The 
sound  and  tight  abstraction  of  dynamics  within  each  domain 
is  critical  to  control  error  when  performing  hybridization. 
The  main  reason  to  consider  alternative  domains  is  in  or¬ 
der  to  reduce  this  error.  For  general  nonlinear  dynamics, 
this  often  requires  solving  constrained  nonlinear  optimiza¬ 
tion  problems,  which  can  be  impossible  in  theory  and  ex¬ 
pensive  in  practice.  For  rectangular  domains,  interval  anal¬ 
ysis  [30]  can  be  used  to  provide  guaranteed  bounds  for  this 
problem.  For  other  types  of  domains,  the  success  of  the 
method  depends  on  the  system  being  analyzed.  For  exam¬ 
ple,  to  perform  the  nonlinear  optimization  step  for  simplicial 
domains,  one  can  use  knowledge  of  the  system’s  Lipschitz 
constant  (which  will  be  sound  but  inaccurate),  or  compute 
bounds  on  the  second  partial  derivatives  (the  elements  of 
the  Hessian  matrix)  [8,9,21].  In  general,  this  is  a  nonlin¬ 
ear  optimization  problem  with  linear  constraints,  but  for 
specific  cases  it  can  be  efficiently  solved.  For  example,  for 
quadratic  dynamics  [20,21],  the  Hessian  matrix  is  constant. 
The  choice  of  domains  is  not  critical  to  the  methods  be¬ 
ing  developed  in  this  paper,  so  for  simplicity,  we  considered 
rectangular  domains. 

A  second  choice  when  performing  hybridization  is  the 
type  of  ‘simpler’  dynamics.  Choices  range  from  constant 
bounds  [16,29,31,32],  linear  and  affine  bounds  [9,21,31], 
to  polynomial  bounds  [1,  18].  In  this  paper,  we  target  an 
unmodified  implementation  of  the  SpaceEx  tool  [24],  and 
therefore  simplify  from  nonlinear  dynamics  to  affine  dynam¬ 
ics. 

3.  PRELIMINARIES 

In  order  to  define  and  justify  the  soundness  of  the  model 
transformation  steps  used  in  our  approach,  we  need  to 
first  precisely  define  the  syntax  and  semantics  of  hybrid 
automata. 

Definition  1.  A  hybrid  automaton  TL  is  defined  by  a  tuple 
TL  =  (Modes,  Var,  Init,  Flow,  Trans,  Inv),  where:  (a)  Modes 
is  a  finite  set  of  modes,  (b)  Var  =  {xi, . . . ,  x„}  is  a  set  of 
real-valued  variables,  (c)  Init(m)  C  Rn  is  the  set  of  initial 
values  for  xi,...,xn  for  each  mode  m  £  Modes,  (d)  For 
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each  m  G  Modes,  the  flow  relation  Flow(m )  is  a  relation 
over  the  variables  in  x  and  their  derivatives  x  =  /m(x), 
where  x(t)  G  R™  and  /  :  R™  — >  2®  ,  i.e.,  differential  inclu¬ 
sions  are  allowed,  (e)  Trans  is  a  set  of  discrete  transitions 
t  =  (m,  g,v,m'),  where  m  and  m!  are  the  source  and  the 
target  modes,  g  is  the  guard  of  t,  and  v  is  the  update  of  t. 
(f)  Inv(m)  C  Rn  is  an  invariant  for  each  mode  m  G  Modes. 

For  a  time  interval  T,  we  define  a  trajectory  of  TL  from 
state  s  =  (m,  x)  to  state  s'  =  (m'.x')  as  a  tuple  (L,X).  In 
this  tuple,  the  function  L  :  T  — >  Modes  and  X  :  T  — >  Rn  are 
functions  that  define  for  each  time  point  in  T  the  mode  and 
values  of  the  continuous  variables,  respectively. 

A  state  s'  is  reachable  from  a  state  s  if  there  exists  a 
trajectory  starting  with  s  and  ending  with  s'.  A  state  s' 
is  reachable  if  s'  is  reachable  from  a  state  s  where  s  is  an 
initial  state.  We  denote  the  set  of  states  reachable  from 
the  set  X  in  mode  m  by  Reach^(m,X).  Reach(H)  of  TL  is 
defined  as  the  set  of  states  that  are  reachable  from  the  set 
of  initial  states.  We  use  Reach^(m,X)  and  Reachc("H)  to 
denote  the  versions  of  the  these  operators  that  return  only 
the  continuous  part  of  the  computed  state  space.  We  refer 
to  Reachfy'H)  as  the  continuous  reachable  state  space  of  TL. 
We  denote  the  projection  of  the  set  R  C  Rn  over  variables 
Var  to  the  subset  Var'  C  Var  by  R  [  var '  ■  Throughout  the 
paper,  we  always  refer  to  time-bounded  reachability,  i.e.,  we 
consider  trajectories  which  evolve  up  to  the  time  horizon 
Tmax-  In  order  to  simplify  notations,  we  implicitly  take  this 
assumption  for  granted  in  our  reasoning.  Finally,  given  a 
mode  m  of  the  automaton  TL,  we  refer  to  the  set  of  outgoing 
transitions  as  Transn(m) . 

4.  TRANSFORMATIONS 

We  are  interested  in  methods  to  compute  an  overapprox¬ 
imation  of  the  time-bounded  set  of  reachable  states,  which 
produce  tight  overapproximations,  yet  are  feasible  from  the 
computational  point  of  view.  The  proposed  approaches  rely 
on  several  hybrid  automaton  transformations.  A  source-to- 
source  transformation  takes  as  input  a  hybrid  automaton  TL, 
a  mode  m  G  Modes,1  possibly  some  additional  parameters, 
and  returns  as  output  another  hybrid  automaton  9(TL).  The 
four  described  transformations  are  (1)  time-triggered  split¬ 
ting,  (2)  space-triggered  splitting,  (3)  domain  contraction, 
and  (4)  dynamics  abstraction.  In  time-triggered  splitting,  a 
given  mode  of  TL  is  split  into  possibly  multiple  modes  via 
a  time-triggered  splitting  of  the  modes.  Similarly,  in  space- 
triggered  splitting,  a  mode  is  split  by  augmenting  the  mode 
invariant  with  a  constraint  induced  by  a  space  trigger  func¬ 
tion.  Domain  contraction  adds  auxiliary  invariants  called 
contraction  domains  to  a  mode  by  intersecting  them  with 
the  existing  invariants  of  the  mode.  Dynamics  abstraction 
overapproximates  the  dynamics  in  a  mode  of  the  automaton, 
which  in  this  paper,  abstracts  nonlinear  differential  equa¬ 
tions  by  linear  differential  inclusions,  in  particular  a  linear 
differential  equation  with  an  additive  set-valued  (interval 
vector)  input. 

As  hybridization  of  the  continuous  dynamics  of  hybrid 
automata  is  the  most  challenging  part  of  the  hybridization 

1For  simplicity  of  presentation,  each  transformation  is  de¬ 
fined  for  a  given  mode  of  the  hybrid  automaton  TL,  and  their 
application  to  multiple  modes  of  TL  is  straightforward  by  it¬ 
erating  over  each  element  of  Modes. 


process,  we  focus  on  the  continuous  dynamics  of  hybrid  sys¬ 
tems  in  the  rest  of  the  paper  and  assume  that  an  input 
hybrid  automaton  has  only  one  mode.  Our  approach  over¬ 
approximates  the  behavior  of  the  original  system  by  a  hy¬ 
brid  automata  consisting  of  multiple  modes.  Therefore,  only 
reachable  continuous  states  are  relevant  for  the  soundness 
of  the  transformations.  This  fact  allows  us  to  to  conclude 
that  the  inclusion  of  the  original  continuous  reachable  state 
space  into  the  transformed  one  is  enough  to  show  sound¬ 
ness  of  our  transformations.  Note,  however,  that  although 
the  input  hybrid  automaton  for  the  whole  hybridization  ap¬ 
proach  is  assumed  to  be  a  singleton,  our  transformations  are 
defined  in  terms  of  general  hybrid  automata. 

In  this  section,  each  of  these  four  transformations  is  pre¬ 
cisely  defined.  After,  these  will  be  combined  in  order  to  per¬ 
form  static  time-triggered  and  mixed-triggered  hybridiza¬ 
tion. 

4.1  Time- Triggered  Splitting 

The  time-triggered  splitting  transformation,  informally, 
separates  the  handling  of  system  behavior  in  the  first  r  time 
units,  and  the  rest  of  the  trajectory  up  to  the  time  hori¬ 
zon.  In  order  to  achieve  this  goal,  the  transformation  splits 
a  given  mode  of  a  hybrid  automaton  into  two  and  imposes 
constraints  that  guarantee  that  the  system  dwells  in  the  first 
mode  for  r  time  units  and  proceeds  to  the  second  one  once 
the  time  threshold  has  been  reached. 

Definition  2.  A  time-triggered  splitting  is  a  transforma¬ 
tion  6tt  of  a  hybrid  automaton  TL,  that  takes  as  input  an 
automaton  TL,  a  mode  m  G  Modes  that  has  no  outgoing 
transitions2,  and  a  real  positive  time  r,  a  time-trigger  thresh¬ 
old.  The  hybrid  automaton  TLtt  =  9tt{TL)  is  defined  as: 
(a)  Modesutt  =  Modesn  U  { mtt },  where  mu  is  a  fresh  (i.e., 
unique)  mode  name,  (b)  Var-utt  =  Var-u  U  {t},  where  t  is 
known  as  the  time-trigger  variable  and  is  fresh,  i.e.,  assume 
without  loss  of  generality  that  t  is  a  unique  variable  name,3 
(c)  the  initial  states  are  copied;  in  addition,  if  Initn{m)  is 
not  the  empty  set  (i.e.,  m  is  an  initial  mode),  then  /mf-Htt(m) 

=  Initu(m )  A  t  =  r,  and  otherwise  Init-ntt(m)  =  Initn(m)', 
InitHttigntt )  =  0,  (d)  the  flows  are  copied,  and  Flow-Htt(mtt) 
=  Flown(m),  so  mode  mtt  copies  the  original  dynamics  of 
m,  and  in  m,  t  =  — 1 ,  and  in  all  modes  other  than  m,  t  =  0, 
(e)  the  transitions  are  copied;  in  addition,  TransHtti.mtt)  = 
Transn  ( m ) ,  with  an  additional  transition  created  from  m  to 
mtt  with  the  guard  t  =  0;  moreover,  every  incoming  tran¬ 
sition  to  m  has  the  reset  t  :=  r  added,  (f)  the  invariants 
are  copied;  in  addition  t  >  0  is  added  to  Invntt(m)  and 
Inv-Htt(rnu)  =  Invn(m)  ( mtt  copied  the  original  invariant  of 
m). 

Figure  1  illustrates  the  time-triggered  splitting  for  a  single 
mode.  A  time-triggered  transition  corresponds  to  any  tran¬ 
sition  with  guard  t  =  0  taken  when  the  time-trigger  variable 

2  In  order  to  make  the  presentation  of  our  transformation 
clearer,  we  consider  a  mode  with  no  outgoing  transitions. 
Our  construction  can  be  easily  generalized  to  also  accom¬ 
modate  this  feature. 

3  If  the  time-triggered  splitting  transformation  dtt  is  applied 
to  an  automaton  multiple  times,  the  time-trigger  variable 
may  be  reused  in  each  splitting,  as  it  needs  only  to  be  fresh 
on  the  first  application  of  the  transformation.  This  opti¬ 
mization  is  done  in  our  implementation. 
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Figure  1:  The  time-triggered  splitting  transformation  applied  to  the  original  automaton  (left,  blue)  produces  the  output 
automaton  (right,  yellow).  An  additional  time-trigger  variable  t  is  added  that  counts  down  to  zero  from  an  initial  time  r. 


t  =  0.  In  contrast  to  general  guards,  the  reachability  along 
time-triggered  transitions  can  be  computed  computationally 
efficient  as  many  reachability  algorithms  automatically  cap¬ 
ture  time  dependencies  as  part  of  their  workflow.  For  ex¬ 
ample,  the  STC  scenario  [23]  of  the  hybrid  model  checker 
SpaceEx  computes  time- dependent  piecewise-linear  approx¬ 
imations  of  the  support  functions  evolution. 

The  following  lemma  connects  the  time-triggered  splitting 
transformation  with  the  original  hybrid  automaton. 

Lemma  4.1.  Let  'LL  be  a  hybrid  automaton  with  a  set  of 
continuous  variables  Var,  m  £  Modes  be  a  mode  without 
outgoing  transitions,  and  r  £  R>o  be  a  time-trigger  thresh¬ 
old.  Then  it  holds  that  Reachc("H)  C  Reachc(0tt('H))  [var- 

Here,  we  note  that  we  need  to  project  away  the  auxiliary 
variable  t  in  order  to  ensure  that  the  sets  of  reachable  states 
of  LI  and  9u(LL)  can  be  compared. 

4.2  Space- Triggered  Splitting 

Space-triggered  splitting,  similar  to  time-triggered  split¬ 
ting,  breaks  a  given  mode  into  several  modes.  However, 
in  contrast  to  the  time-triggered  transformation,  it  uses  a 
space-trigger  function  to  define  criteria  for  mode  splitting. 

Definition  3.  A  space-triggered  splitting  is  a  transforma¬ 
tion  9st  of  a  hybrid  automaton  LL,  that  takes  as  input  an 
automaton  LL,  a  mode  m  £  Modes  that  has  no  outgoing 
transitions,  and  a  function  7r  :  Rn  — R  called  the  space- 
trigger  function.  The  function  n  must  satisfy  the  condition 
that  upon  entering  mode  m,  7r(a:)  >  0,  where  x  is  the  cur¬ 
rent  state.  This  means  that  if  m  is  an  initial  mode,  for  all 
states  x  £  Init(m),  iv(x)  >  0.  The  hybrid  automaton  List  = 
9st(LL )  defined  as:  (a)  Modes-uat  =  Modes-u  U  {mst},  where 
mst  is  a  fresh  (i.e.,  unique)  mode  name,  (b)  Var-ust  =  Var-u, 
(c)  the  initial  states  are  copied;  Init-Hat(mst)  =  0,  (d)  the 
flows  are  copied;  in  addition,  Flown st(mst)  =  Flown(m), 
(e)  the  transitions  are  copied;  in  addition,  Transnst  (mst) 
=  Transn{m);  moreover,  an  additional  transition  created 
from  m  to  mst  with  the  guard  tt(x)  =  0,  and  (f)  the  in¬ 
variants  are  copied,  with  7r(a:)  >  0  added  to  Inv-uat(m)  and 

Invnat(mst)  =  Invn(m)  ( mst  copied  the  original  invariant 
of  m ). 

The  space-triggered  splitting  transformation  adapts  the 
idea  of  pseudo-invariants  [11]  to  the  hybridization  setting. 
In  our  setting,  a  space-trigger  function  7r  basically  plays  a 
role  of  a  pseudo-invariant. 

The  resulting  automaton  overapproximates  the  continu¬ 
ous  reachable  state  space  of  the  original  one  which  is  for¬ 
mally  stated  in  the  following  lemma. 


Lemma  4.2.  Let  LL  be  a  hybrid  automaton,  m  £  Modes 
be  a  mode  without  outgoing  transitions,  and  n  :  R"  — >  R  be 
a  function  satisfying  the  assumptions  in  Definition  3.  Then 
Reach" (LL)  C  Reach" (9st(LL)) . 

4.3  Domain  Contraction 

Domain  contraction  adds  auxiliary  invariants  known  as 
contraction  domains  that  should  contain  the  set  of  reachable 
states.  Given  a  set  D  and  a  mode  m  of  a  hybrid  automaton 
LL  where  x  =  fm(x),  if  Reach^m,  X )  C  D  for  X  C  Inv(m), 
i.e.  the  set  of  reachable  states  from  mode  m  starting  from  a 
subset  A'  C  Inv(m )  is  contained  in  D,  then  D  may  safely  be 
added  as  an  invariant  of  m.  Of  course,  the  set  of  reachable 
states  is  not  available  and  is  what  is  being  computed  or 
approximated,  so  error  modes  known  as  domain  contraction 
error  modes  (DCEMs)  are  used  to  maintain  soundness  if 
the  system  leaves  the  states  represented  by  these  auxiliary 
invariants. 

Definition  f.  A  domain  contraction  is  a  transformation 
9dc  of  a  hybrid  automaton  LL,  that  takes  as  input  an  au¬ 
tomaton  LL,  a  mode  m  £  Modes,  and  a  set  D  C  R™  called 
the  contraction  domain  auxiliary  invariant. 

The  transformed  hybrid  automaton  Lidc  =  9dc(LL)  is  de¬ 
fined  as:  (a)  Modesudc  =  Modes-u  U  {err},  the  modes  are  the 
copied,  with  a  new  domain  contraction  error  mode  (DCEM) 

err  added,  (b)  Var-udc  =  Var-u,  (c)  the  initial  states  are 
copied;  additionally,  if  m  is  an  initial  mode,  and  Init(m)  is 
not  entirely  contained  in  D,  then  add  the  err  DCEM  to  the 
initial  states;  in  this  way,  we  capture  a  degenerate  case  if 
the  initial  set  has  states  outside  of  the  contraction  domain, 
(d)  the  flows  are  copied;  additionally,  Flow-udc(err)  of  the 
form  x  =  0  are  added,  (e)  the  transitions  are  copied,  with 
additional  transformations  of  the  following  form:  given  an 
incoming  transition  d  =  (n,  g,v,m)  to  mode  m  in  Li,  (1) 
augment  the  guard  of  the  transition  d  with  x  £  D,  and  (2) 
add  an  additional  transition  d1  =  (n,g  A  x  £  cl(D),err) 
with  an  extra  condition  x  £  cl(D)  on  the  guard  and  leading 
to  the  DCEM  err,  where  D  denotes  the  complement  of  D 
and  cl(-)  stands  for  topological  closure  and  (3)  add  an  addi¬ 
tional  transition  d"  =  (m,x  £  cl(D),  err),  (f)  the  invariants 
are  copied,  except  for  the  invariant  Inv-Hdc(m )  =  Inv-u(m) 
n  x  £  D. 

A  visualization  of  the  domain  contraction  transformation 
is  given  in  Figure  2. 

The  conditions  to  enter  a  DCEM  together  ensure  that 
regardless  of  the  choice  of  the  contraction  domain,  if  the 
DCEM  err  is  not  reached,  then  the  overapproximation  of  the 
reachable  states  is  sound.  Additionally,  the  condition  that 
the  dynamics  are  zero  in  the  DCEM  err  ensures  that  during 
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Figure  2:  The  domain  contraction  transformation  applied  to  the  original  automaton  (left,  blue)  produces  the  output  automa¬ 
tion  (right,  yellow).  The  contraction  domain  D  is  added  to  the  invariant,  with  DCEM  err  inserted  to  detect  if  the  reachable 
set  of  states  leaves  D. 


a  reachability  computation,  the  exploration  of  the  err  will 
terminate  and  be  a  dead-end  in  the  exploration  of  the  state- 
space.  Note  that  the  notion  of  topological  closure  is  required 
to  ensure  that  the  intersection  of  guard  and  invariant  is  non¬ 
empty. 

Lemma  4.3.  Let  Tl  be  a  hybrid  automaton,  m  £  Modes 
be  a  mode,  and  D  C  R71  be  a  contraction  domain.  Then,  if 
no  DCEM  is  reachable,  Reach"  (%)  C  Reach c(9dc{Tl)). 

The  contraction  domain  auxiliary  invariants  may  be  arbi¬ 
trary  and  may  be  determined  using  any  method,  so  they  may 
not  actually  contain  the  set  of  reachable  states.  To  main¬ 
tain  soundness,  the  DCEMs  are  added  such  that  if  the  con¬ 
traction  domains  do  not  contain  the  set  of  reachable  states, 
transitions  to  the  DCEMs  may  be  taken.4  If  no  DCEMs 
are  reached,  then  the  domain  contraction  transformation  is 
sound,  but  otherwise,  if  a  DCEM  is  reached,  the  resultant 
set  of  set  of  reachable  states  may  not  be  subset  of  the  original 
automaton’s  set  of  reachable  states.  If  it  is  known  that  the 
set  of  reachable  states  will  not  leave  the  contraction  domain 
by  some  other  analysis,  then  the  DCEMs  are  not  necessary 
and  the  invariants  may  simply  be  augmented  (conjuncted) 
with  the  contraction  domain.  In  summary,  if  the  contrac¬ 
tion  domains  do  not  contain  the  set  of  reachable  states  for  a 
given  mode,  then  a  state  with  a  mode  equal  to  the  DCEM 
will  be  reached. 

4.4  Dynamics  Abstraction 

Continuous  dynamics  are  abstracted  by  transforming  the 
flows  of  the  original  hybrid  automaton  into  flows  with  in¬ 
creased  nondeterminism.  In  this  paper,  nonlinear  differ¬ 
ential  inclusions  are  overapproximated  using  linear  differ¬ 
ential  inclusions,  specifically  linear  ODEs  with  an  additive 
set- valued  input. 

Definition  5.  A  dynamics  abstraction  is  a  transformation 
9 da  of  a  hybrid  automaton  TL,  that  takes  as  input  an  automa¬ 
ton  TL,  a  mode  m  £  Modes,  and  a  set-valued  function  g  : 
R™  -A  2r  called  the  abstract  dynamics,  where,  for  the  flow 
x  =  fm(x)  of  mode  m  with  invariant  Inv(m),  gm(x)  is  such 
that  Vx  £  Inv(m):  /m(x)  C  gm(x).  The  hybrid  automa¬ 
ton  Ttda  =  9da(TL)  is  defined  as:  (a)  Modesnda  =  Modesu, 

(b)  Varjida  =  Vary,,  (c)  the  initial  states  are  copied,  (d)  the 
flows  are  copied,  except  for  Flowyda(m)  which  is  set  to 
x  =  g(x),  (e)  the  transitions  are  copied,  (f)  the  invariants 
are  copied. 

4  Since  the  semantics  of  hybrid  automata  defined  do  not  sup¬ 
port  urgency  or  must  transitions,  we  exploit  the  fact  that 
the  reachability  computation  explores  all  paths  to  ensure 
soundness. 


Similarly  to  other  transformations  we  have  considered,  we 
formulate  a  lemma  relating  the  original  and  transformed  sys¬ 
tems. 

Lemma  4.4.  Let  TL  be  a  hybrid  automaton,  m  £  Modes 
be  a  mode,  g  :  R”  — >  21  be  a  set-valued  function  satis¬ 
fying  the  assumptions  in  Definition  5.  Then  it  holds  that 
Reach"  (%)  C  Reach"  (^(W)). 

5.  MIXED-TRIGGERED  HYBRIDIZATION 

Now  we  present  the  central  result  of  the  paper,  a  static 
mixed-triggered  hybridization  that  combines  the  four  trans¬ 
formations  we  have  introduced. 

Definition  6.  A  static  mixed-triggered  hybridization  is  a 
transformation  9mt.  of  a  hybrid  automaton  Tl  and  has  the 
following  input: 

•  a  single-mode  automaton  Tl, 

•  a  list  of  splitting  elements  E\....En_ i,  where  each 
element  Ei  is  either  a  real  number  to  be  used  for  time- 
triggered  splitting,  or  a  n  function  to  be  used  for  space- 
triggered  splitting  (list  1), 

•  D i , . . . ,  Dn  are  the  contraction  domains  (sets)  for  each 
new  location  (list  2),  and 

•  g  \ , ,  gn  are  the  dynamics  abstraction  functions  for 
each  location  (list  3). 

The  mixed-triggered  hybridization  transformation  con¬ 
sists  of  the  following  three  steps: 

•  Apply  either  time-triggered  splitting  or  space-triggered 
splitting  based  on  the  list  E\ . . . .  En- 1.  We  apply  each 
transformation  to  the  most-recently  constructed  mode, 
which  has  no  outgoing  transitions.  The  result  of  this 
step  is  a  chain  of  modes. 

•  For  each  mode  in  the  chain,  apply  N  domain  contrac¬ 
tions  based  on  the  list  D\, ... ,  D„. 

•  For  each  mode  in  the  chain,  apply  N  dynamics  ab¬ 
stractions  based  on  the  list  gi, ...  ,gn. 

If  the  list  of  splitting  elements  (list  1)  contains  only  time- 
triggered  splitting  elements  (and  no  space-triggered  splitting 
elements),  then  it  is  a  static  time-triggered  hybridization. 

The  following  theorem  establishes  the  soundness  of  the 
mixed-triggered  hybridization. 

Theorem  5.1.  For  hybrid  automaton  Tl,  if  no  DCEM  are 
reachable,  then  the  continuous  reachable  state  space  of  the 
mixed-triggered  transformation  9mt(Tl)  overapproximates  the 
continuous  reachable  state  space  of  the  original  automaton: 
Reach"  (H)  C  Reach"  (6>mi(«)). 
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Proof.  The  proof  follows  by  a  straight-forward  applica¬ 
tion  of  Lemmas  4.1,  4.2,  4.3,  and  4.4.  □ 

We  observe  that  the  mixed-triggered  hybridization  ap¬ 
proach  contains  a  number  of  parameters  which  must  be 
carefully  chosen  in  order  to  guarantee  a  sound  abstraction, 
which  is  ensured  when  no  error  modes  (DCEMs)  are  reach¬ 
able.  If  the  contraction  domains  are  too  small,  then  the  set 
of  reachable  states  will  exit  the  domain  and  the  DCEM  will 
be  reached.  If  the  contraction  domains  are  too  large,  then 
the  dynamics  abstraction  will  be  a  large  overapproximation, 
and  the  set  of  reachable  states  will  become  both  large  and 
inaccurate.  In  modes  copied  during  time-triggered  splitting, 
whenever  the  time-triggered  variable  t  reaches  zero,  the  set 
of  reachable  states  at  each  mode  must  be  contained  in  the 
domains  (invariants)  of  both  the  source  and  target  locations. 
Space-triggered  splitting  requires  as  input  the  7r  functions 
which  determine  the  splitting  structure. 

In  the  following,  we  describe  an  approach  to  generate  the 
parameters  for  proposed  hybridization  approach  in  a  way 
that  will  satisfy  the  above  requirements.  Again,  the  ap¬ 
proach  is  described  assuming  a  single-location  hybrid  au¬ 
tomaton,  where  the  initial  set  of  states  is  a  rectangle,  al¬ 
though  generalizations  are  not  difficult. 

5.1  Parameter  Selection  Algorithm 

In  order  to  construct  the  three  lists  to  be  used  as  hy¬ 
bridization  parameters,  an  algorithm  is  proposed  which  uses 
numerical  simulations.  The  proposed  approach  has  its  own 
user-provided  parameters: 

•  T  is  the  maximum  time, 

•  S  a  simulation  strategy,  one  of  {point,  star,  STAR- 
CORNERS} 

•  Stt  is  the  simulation  time  in  a  time-triggered  transfor¬ 
mation  step, 

•  npi  is  the  number  of  space-triggered  transformation 
steps  to  use, 

•  Spi  is  the  maximum  simulation  time  when  performing 
a  space-triggered  transformation  step, 

•  e  is  a  bloating  term  to  account  for  the  difference  be¬ 
tween  the  simulated  points  the  set  of  reachable  states. 

The  algorithm  first  selects  a  finite  set  of  simulation  points 
sampled  from  the  initial  set  of  states.  If  S  is  point,  only  the 
center  of  the  initial  rectangle  is  used.  If  5  is  STAR,  the  center 
is  used,  as  well  as  the  center  of  every  face  of  the  rectangle, 
1  +  2n  points,  where  n  is  the  number  of  variables.  If  S  is 
STARCORNERS,  the  center  is  used,  as  well  as  the  centers  of 
every  face,  as  well  as  the  corners  of  the  initial  rectangle, 
1  +  2n  +  2n  points.  Selecting  more  points  may  permit  a 
smaller  e,  but  since  the  number  of  points  is  exponential,  the 
STARCORNERS  strategy  may  not  always  be  practical.  The 
collection  of  points  are  stored  in  a  variable,  sims. 

The  algorithm  proceeds  in  iterations,  at  each  iteration 
doing  either  a  space-triggered,  step,  or  a  time-triggered  step. 
The  three  parameter  lists  (the  output)  are  initially  empty.  A 
current  time  variable  ct,  initially  zero,  is  maintained  which 
tracks  the  amount  of  time  elapsed  during  time-triggered 
steps  (space-triggered  steps  do  not  add  to  ct).  A  second 
variable  next_st  tracks  the  time  at  which  to  insert  the  next 
space-triggered  value.  If  npi  >  0,  next_st  is  initialized  to  0, 
otherwise  it  is  set  to  oo. 

At  each  iteration,  if  the  current  time  ct  variable  is  greater 
than  or  equal  to  next  space-triggered  time  variable  next_st, 


a  space-triggered  step  is  attempted  and  next_st  is  increased 
by  U — .  Otherwise,  a  time-triggered  transition  is  performed 
and  ct  is  increased  by  Stt-  The  process  completes  when  ct 
exceeds  the  maximum  time  T. 

A  time-triggered  step  adds  Stt  to  output  list  1.  Then, 
it  computes  the  bounding  box  of  sims,  bloats  it  by  e,  and 
stores  it  in  start.  Each  point  in  sims  is  numerically  simu¬ 
lated  for  Stt  time.  The  bounding  box  of  sims  is  computed 
again,  bloated  by  e,  and  stored  in  end.  The  bounding  box  of 
start  and  end  is  then  computed,  and  put  into  output  list  2 
(contraction  domains) . 

A  visualization  of  two  consecutive  time-triggered  steps  is 
shown  in  Figure  3.  Here,  S  =  point,  so  sims  is  just  a 
single  point.  Initially,  sims  is  a.  After  Stt  time,  the  point 
/3  is  reached;  after  Stt  further  time,  the  simulation  reaches 
7.  The  modification  of  the  output  lists  after  these  two  steps 
would  be  the  time-triggered  value  Stt  twice  inserted  into  list 
1,  the  red  rectangle  set  inserted  into  list  2,  followed  by  the 
green  rectangle  set  inserted  into  list  2. 

A  space-triggered  step  attempts  to  use  numerical  sim¬ 
ulations  to  find  a  function  7r  for  space-triggered  splitting, 
but  may,  in  certain  cases,  be  aborted  without  modifying  the 
output  lists.  First,  the  bounding  box  of  sims  is  computed, 
bloated  by  e,  and  stored  in  start.  The  center  point  in  sims, 
which  we  call  p,  is  numerically  simulated  until  either,  (1)  the 
plane  induced  by  the  point  lies  entirely  on  one  side  of  start, 
or  (2)  the  space-triggered  time  limit  5pi  is  reached.  If  con¬ 
dition  (2)  occurs,  the  space-triggered  step  returns  without 
modifying  the  output  lists,  and  reverts  the  status  of  sims. 
For  condition  (1),  the  plane  induced  by  a  point  p  is  a  hyper¬ 
plane  that  both  contains  p  and  is  orthogonal  to  the  gradient 
at  p.  The  function  n  is  created  from  the  equation  of  the 
hyperplane,  where  n  is  zero  along  the  plane  and  positive  on 
the  side  of  start  (in  the  opposite  direction  of  the  gradient 
at  p).  Forcing  transitions  along  hyperplanes  orthogonal  to 
the  gradient  was  previously  shown  as  effective  in  reducing 
the  size  of  the  currently-tracked  set  of  reachable  states  in 
the  context  of  pseudo-invariants  [11,12].  Each  of  the  other 
points  in  sims  are  then  numerically  simulated  until  either  (1) 
they  reach  a  point  along  the  constructed  hyperplane  where 
7r  evaluates  to  zero,  or  (2)  they  are  simulated  for  the  space- 
triggered  time  limit  5pi.  If  for  any  point  condition  (2)  occurs, 
again,  the  space-triggered  step  aborts  without  modifying  the 
output  lists,  and  reverts  the  status  of  sims.  If  condition  (1) 
occurs  for  every  point  in  sims,  the  bounding  box  of  all  the 
points  in  sims  (which  are  all  along  the  hyperplane)  is  taken, 
bloated  by  e,  and  assigned  to  end.  The  bounding  box  of 
start  and  end  is  then  computed,  and  put  into  output  list 
2  (contraction  domains).  The  hyperplane  function  7r  is  put 
into  output  list  1. 

At  the  end  of  the  iterative  construction,  output  list  3  is 
created  by  performing  linearization  in  each  of  the  contrac¬ 
tion  domains  in  list  2,  and  then  solving  for  the  difference 
between  the  nonlinear  dynamics  function  and  its  lineariza¬ 
tion.  This  is,  in  general,  a  global  optimization  problem, 
although  guaranteed  bounds  can  be  computed  using,  for  ex¬ 
ample,  interval  arithmetic.  This  is  also  an  embarrassingly 
parallel  problem,  which  can  be  exploited  to  speed  up  this 
computationally  expensive  step. 

Finally,  the  last  element  of  list  1  is  removed,  so  that  the 
last  mode  in  the  constructed  chain  will  not  be  split.  This 
process  results  in  three  lists,  the  first  of  size  N  —  1,  and  the 
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Figure  3:  Two  time-triggered  steps  use  numerical  simula¬ 
tions  to  create  two  contraction  domains. 

other  two  of  size  N,  as  is  needed  by  the  proposed  mixed- 
triggered  hybridization  approach. 

5.2  Generalizations 

The  proposed  construction  approach  is  simple  in  that  only 
a  small  number  of  user-parameters  are  required.  However, 
fine-tuning  is  possible  which  can  create  more  precise  abstrac¬ 
tions,  at  the  cost  of  requiring  more  input  from  the  user. 

First,  the  time  step  Stt  could  be  changed  for  each  domain. 
In  Figure  3  this  would  correspond  to  the  case  where  the  dif¬ 
ference  in  simulation  times  between  points  a  and  (3  is  not 
the  same  as  the  difference  between  /3  and  7.  Next,  a  per- 
domain  bloating  term  e  is  possible.  Furthermore,  each  do¬ 
main’s  bloating  term  could  be  further  parameterized  based 
on  the  face  of  the  rectangular  domain. 

The  domains  need  not  be  rectangles  aligned  to  axes.  Do¬ 
mains  which  are  rotated  rectangles,  aligned  with  the  direc¬ 
tion  of  the  flow,  could  reduce  the  error  in  the  dynamics 
abstraction  step.  As  with  other  hybridization  work  [21],  do¬ 
mains  which  are  triangles  (simplices),  or  rotated  variants 
could  also  be  used.  The  complication  with  these  approaches 
is  that  the  global  optimization  step  of  domain  abstraction, 
which  is  necessary  for  soundness,  can  become  more  compli¬ 
cated.  For  example,  the  simplex-based  approach  requires 
optimizing  the  Hessian  matrix  of  the  dynamics  in  a  simplex 
domain,  which  may  be  difficult  depending  on  the  specific 
location’s  dynamics. 

6.  EVALUATION 

As  stated  by  Theorem  5.1,  in  order  to  soundly  reason 
about  the  set  of  reachable  states  of  the  original  automa¬ 
ton,  the  output  automaton  from  the  mixed-triggered  hy¬ 
bridization  process  must  not  reach  any  DCEMs.  The  main 
purpose  of  the  evaluation,  therefore,  is  (1)  to  demonstrate 
that  the  hybridization  parameters  derived  from  simulations 
can  result  in  models  where  DCEMs  are  not  reached  during 
reachability  analysis  of  the  output  automaton.  Additionally, 
we  aim  to  (2)  demonstrate  the  benefits  of  occasional  space- 
triggered  transitions  compared  with  a  pure  time-triggered 
approach.  Finally,  we  (3)  demonstrate  improved  scalability 
by  running  our  developed  static  approach  on  a  higher  dimen¬ 
sional  model,  at  a  granularity  that  would  be  impossible  for 
existing  static  approaches.  The  evaluation  was  performed 
with  these  three  goals  in  mind. 

The  proposed  hybridization  method  was  implemented  in 
the  Hyst  model  translation  and  transformation  tool  [12]°. 
The  developed  transformation  pass  implements  the  algo¬ 
rithm  described  in  Section  5  leveraging  the  transformations 

5SpaceEx  model  files  for  the  examples  evaluated,  both 
before  and  after  hybridization,  are  available  at:  http:// 
veri  vital,  com /hyst  /  pass-  hybridization  / . 


(a)  Computed  reachability  (b)  Streamplot 

Figure  4:  The  limit  cycle  for  the  Van  der  Pol  system  was 
computed  with  SpaceEx  using  our  hybridization  approach. 

of  Section  4.  We  target  the  latest  version  of  the  SpaceEx 
tool,  which  supports  time-triggered  transitions  using  the 
map-zero-duration- jump-sets  flag.  In  order  to  derive  the 
dynamics  abstraction  function,  we  use  a  global  optimiza¬ 
tion  routine  from  the  scipy.  optimize  library.  Other  op¬ 
tions  are  possible,  for  example  interval  arithmetic,  interval 
arithmetic  with  grid-paving,  SMT  solvers,  or  combinations 
of  these  methods.  Since  the  optimizations  in  each  domain 
are  run  in  parallel,  more  effort  can  be  taken  to  derive  tighter 
bounds  without  significant  effects  on  overall  runtime.  The 
reported  times  were  measured  on  a  computer  with  an  Intel 
Core  2  Quad  CPU  (Q9650)  at  3.00  GHz  with  4  GB  RAM. 

6.1  Van  der  Pol  Oscillator 

The  first  set  of  experiments  consider  a  Van  der  Pol  oscil¬ 
lator,  which  is  a  two-dimensional  system  with  the  following 
nonlinear  dynamics: 

x  =  y 

y  =  (1  —  x2)  *  y  —  x 

We  use  the  same  initial  states  as  evaluated  in  other  hy¬ 
bridization  approaches  [1],  ( x,y )  €  [1.25,1.55]  x  [2.28,2.32]. 
A  maximum  time  of  5.5  was  used,  which  is  sufficient  to  com¬ 
plete  one  cycle  of  the  oscillator,  as  in  the  earlier  work. 

We  used  numerical  simulations  based  on  the  S  =  STAR 
strategy,  a  time-triggered  step  of  Stt  =0.05,  a  bloating  term 
of  e  =  0.05,  a  number  of  space-triggered  transformation 
steps  of  npi  =  31,  and  a  maximum  simulation  time  in  a 
space-triggered  transformation  step  of  Spi  =  1.  Analyzing 
the  generated  model  with  SpaceEx  resulted  in  no  DCEMs 
being  reached,  which  means  that  the  set  of  reachable  states 
overapproximates  the  set  of  reachable  states  in  the  the  orig¬ 
inal  automaton.  This  demonstrates  goal  (1)  of  the  evalua¬ 
tion.  The  combined  hybridization  and  computation  process 
took  10.3  seconds.  A  visualization  of  the  resultant  set  of 
reachable  states  produced  by  SpaceEx  is  given  in  Figure  4a, 
and  can  be  compared  to  a  streamplot  of  the  dynamics  given 
in  Figure  4b. 

It  is  insightful  to  examine  the  bounding  box  of  the  numer¬ 
ical  simulations  upon  entering  each  mode,  and  compare  it  to 
the  bounding  box  of  the  set  of  reachable  states  at  the  same 
times.  In  particular,  by  looking  at  the  maximum  width  in 
any  dimension  of  the  bounding  box  of  sims  and  comparing 
it  with  the  maximum  width  of  bounding  box  of  the  set  of 
reachable  states,  we  can  estimate  how  close  the  set  of  reach¬ 
able  states  was  to  the  boundaries  of  the  contraction  domains 
where  a  DCEM  would  be  reached.  A  plot  of  these  widths 
upon  entering  each  mode  is  shown  in  Figure  5. 
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Figure  5:  The  maximum  width  of  the  bounding  boxes  of  the 
reachable  states  and  simulations  upon  entering  each  mode 
remains  within  2  *  e  =  0.1,  which  is  necessary  to  avoid  en¬ 
tering  a  DCEM. 


Figure  6:  Space-triggered  transitions  serve  to  reduce  the  size 
of  the  tracked  set  of  states. 

Since  we  used  a  bloating  term  of  e  =  0.05,  it  is  nec¬ 
essary  that  maximum  width  of  the  simulated  states  plus 
2  *  e  =  0.1  is  greater  than  the  maximum  width  of  the  set  of 
reachable  states  at  all  times,  otherwise,  an  error  state  will 
be  reached.  Additionally,  from  the  plot  we  can  see  that  the 
STARCORNERS  strategy  has  slightly  better  tracking  of  widths 
of  the  set  of  reachable  states  near  the  start  of  the  computa¬ 
tion,  although  it  makes  less  of  a  difference  later  on. 

In  order  to  show  the  effect  of  space-triggered  transitions, 
we  consider  the  same  system  using  a  shorter  time  bound 
of  2.0,  a  time  step  of  Stt  =  0.01,  and  the  same  value  of 
e  =  0.05.  We  run  the  system  with  no  space-triggered  tran¬ 
sitions,  a  single  space-triggered  transition  at  the  start,  and 
four  space- triggered  transitions.  The  widths  of  the  tracked 
set  of  reachable  states,  and  the  bounding  box  of  the  simu¬ 
lated  points  is  shown  in  Figure  6.  Without  space-triggered 
transitions,  the  width  of  the  set  of  reachable  states  quickly 
gets  larger  than  the  simulated  bounding  box,  and  around 
time  0.29,  a  DCEM  is  reached.  With  a  single  space-triggered 
transition  at  the  start,  the  tracked  set  of  states  is  smaller, 
and  a  DCEM  is  not  reached  until  around  time  1.66.  With 
four  space-triggered  transitions,  the  full  2.0  seconds  is  com¬ 
puted  without  reaching  a  DCEM.  Furthermore,  the  decrease 
in  the  size  of  the  tracked  states  is  apparent  at  the  space- 
triggered  times  0.0  (mode  #0),  0.5  (mode  #51),  1.0  (mode 
#102),  and  1.5  (mode  #153).  This  demonstrates  the  effec¬ 
tiveness  of  space-triggered  transitions  in  reducing  the  size  of 
the  currently-tracked  set  of  states,  goal  (2)  of  the  evaluation. 

6.2  Nonlinear  Water  Tank 

The  next  model  we  consider  is  a  nonlinear  tank  model  [5]. 
This  model  is  parameterized  on  the  number  of  tanks,  n, 


(a)  Computed  Reachability  (b)  Result  from  [5]  (includes 
input  disturbances) 

Figure  7:  A  plot  of  a  projection  of  the  computed  reachable 
states  for  xi  and  X2  for  the  6-D  non-linear  tank  model. 

where  we  use  n  =  6.  Each  tank  i  adds  a  single  variable  Xi  to 
the  model,  which  represents  the  height  of  the  water  in  the 
tank.  The  input  to  the  first  tank  is  based  on  the  level  of 
the  last  tank,  xn .  We  analyze  a  deterministic  version  of  the 
model,  with  no  disturbance  input  and  fixed  tank  parameters. 
The  dynamics  for  x\  and  every  other  Xi>  i  are: 

xi  =  0.1  +  0.01(4  —  xn)  +  0.015^'2gxi 
it  =  0.015  \/2gxiZ[  —  0.015 -y/2  gxi 

We  used  the  same  initial  set  of  states  as  the  earlier  work, 
xi  £  [1.9, 2.1],  x2  £  [3.9, 4.1],  x3  £  [3.9, 4.1],  x4  £  [1.9, 2.1], 
X5  £  [9.9,10.1],  and  xe  £  [3.9, 4.1].  Using  the  simulation 
strategy  S  =  STARCORNERS,  a  maximum  time  of  T  =  400,  a 
step  size  of  Stt  =  4,  a  bloating  term  value  of  e  =  0.2,  a  num¬ 
ber  of  space-triggered  transformation  steps  of  npi  =  10,  and 
a  maximum  simulation  time  in  a  space-triggered  transfor¬ 
mation  step  of  5pi  =  10,  the  hybridized  model  was  created. 
SpaceEx  was  used  to  analyze  this  model,  and  indicated  that 
no  DCEMs  were  reached.  The  whole  process  took  about  430 
seconds.  Figure  7  shows  a  projection  of  the  set  of  reachable 
states  onto  x\  and  X2,  as  well  as  a  result  from  the  earlier 
hybridization  work. 

This  demonstrates  goal  (3)  of  the  evaluation,  that  static- 
based  hybridization  approaches  can  scale  to  higher  dimen¬ 
sions.  Although  only  a  six-dimensional  model  was  consid¬ 
ered,  this  is  higher  than  we  could  find  for  any  published 
static  hybridization  method. 

7.  CONCLUSION 

In  this  paper,  we  developed  the  first  static  time-triggered 
and  mixed-triggered  hybridization  approaches.  The  devel¬ 
oped  methods  use  simulations  to  guide  the  hybridization 
process  and  modify  an  input  model  for  analysis  with  off-the- 
shelf  verification  tools,  unlike  dynamic  hybridization  meth¬ 
ods  that  require  tool  modification.  Additionally,  we  can  per¬ 
form  the  expensive  dynamics  abstraction  (linearization)  step 
for  each  mode  in  parallel,  which  can  improve  the  speed  of 
the  method.  We  have  shown  the  effectiveness  of  the  method 
by  hybridizing  example  nonlinear  systems  and  computing 
the  set  of  reachable  states  using  SpaceEx,  a  tool  that  is  only 
capable  of  reasoning  with  linear  and  affine  systems. 

Since  this  is  the  first  paper  investigating  this  category  of 
hybridization  techniques,  we  believe  significant  further  op¬ 
timization  is  possible.  Extending  the  approach  from  single¬ 
mode  input  automata  to  multiple-mode  systems  would  be 
a  straightforward  enhancement,  and  has  been  done  in  other 
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hybridization  approaches  [1].  Dynamic  mixed-triggered  ap¬ 
proaches,  have  also  yet  to  be  investigated.  Parameter  se¬ 
lection  for  the  approach  can  also  be  challenging  and  could 
be  further  automated,  perhaps  by  using  a  CEGAR-like  ap¬ 
proach  to  detect  when  DCEMs  (error  modes)  are  reached, 
and  performing  additional  simulations  from  violation  re¬ 
gions.  Finally,  the  simulation-based  parameter  construc¬ 
tion  algorithm  does  not  track  the  set  of  reachable  states 
well  when  nondeterminism  or  disturbances  are  present,  and 
other  approaches  from  hybrid  automaton  falsification  may 
work  better  in  these  cases. 
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Abstract  Hybrid  automata  are  an  important  formal¬ 
ism  for  modeling  dynamical  systems  exhibiting  mixed 
discrete-continuous  behavior  such  as  control  systems  and 
are  amenable  to  formal  verification.  However,  hybrid  au¬ 
tomata  lack  expressiveness  compared  to  integrated 
model-based  design  (MBD)  frameworks  such  as  the 
MathWorks’  Simulink/Stateflow  (SLSF).  In  this  paper, 
we  propose  a  technique  for  correct-by-construction  com¬ 
positional  design  of  cyber-physical  systems  (CPS)  by 
embedding  hybrid  automata  into  SLSF  models.  Hybrid 
automata  are  first  verified  using  verification  tools  such 
as  SpaceEx,  and  then  automatically  translated  to  em¬ 
bed  the  hybrid  automata  into  SLSF  models  such  that 
the  properties  verified  are  transferred  and  maintained  in 
the  translated  SLSF  model.  The  resultant  SLSF  model 
can  then  be  used  for  automatic  code  generation  and  de¬ 
ployment  to  hardware,  resulting  in  an  implementation. 
The  approach  is  implemented  in  a  software  tool  build¬ 
ing  on  the  HyST  model  transformation  tool  for  hybrid 
systems.  We  show  the  effectiveness  of  our  approach  on  a 
CPS  case  study — a  closed-loop  buck  converter — and  val¬ 
idate  the  overall  correct-by-construction  methodology: 
from  formal  verification  to  implementation  in  hardware 
controlling  an  actual  physical  plant. 


1  Introduction 

In  this  paper,  we  present  the  theory  and  associated  im¬ 
plementation  for  the  translation  of  hybrid  automaton 
models  (used  for  verification)  to  the  MathWorks  Simu¬ 
link/Stateflow  (SlSf)  models,  subsequently  used  for  de¬ 
sign  refinement,  simulation,  implementation,  and  code 
generation  for  target  embedded  hardware.  Our  approach 
is  particularly  useful  if  the  design  process  is  structured 

DISTRIBUTION  A.  Approved  for  public  release;  Distribution 
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Figure  1:  High-level  overview  of  the  model-based  de¬ 
sign  process  enabled  by  this  work.  Verification  using  the 
hybrid  automaton  is  first  performed  in  a  hybrid  sys¬ 
tems  model  checker,  then  we  automatically  generate  a 
trajectory-equivalent  SlSf  diagram.  The  diagram  can 
then  be  embedded  into  a  more  complex  system,  possi¬ 
bly  with  other,  unverified,  components  (because  they  are 
too  large  to  verify,  exist  for  legacy  reasons,  etc.),  and  can 
then  be  used  for  code  generation  and  implementation  in 
actual  systems. 


in  a  bottom-up  fashion.  In  other  words,  we  assume  that 
the  individual  system  components  are  first  modeled  in 
detail,  such  as  modeling  a  control  algorithm  as  a  hybrid 
automaton  and  verifying  properties  (typically  safety)  for 
it.  These  components  are  then  linked  together  to  form 
the  whole  system  under  consideration  within  SlSf.  This 
leads  to  overall  system  models  consisting  of  heteroge¬ 
neous  components  where  a  number  of  components  are 
modeled  as  hybrid  automata,  but  the  entire  system  may 
be  too  complex  to  formally  model  and  verify.  In  the 
last  decade,  a  number  of  powerful  formal  design,  anal¬ 
ysis,  and  verification  tools  for  hybrid  automata  such  as 
SpaceEx  [&-12  22  and  Flow*  17  have  emerged.  In  our 
proposed  approach,  a  designer  can  ensure  the  correct¬ 
ness  of  individual  components  before  using  our  transla¬ 
tion  process  to  link  the  system  together  in  SlSf  (see 

Fig-0- 
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We  introduce  a  technique  to  automatically  convert 
the  hybrid  automata  into  trajectory-equivalent  SlSf  di¬ 
agrams.  By  trajectory-equivalent,  we  mean  that  behav¬ 
iors  (trajectories)  of  the  translated  SlSf  diagram  match 
those  of  the  original  hybrid  automaton.  One  technical 
challenge  is  that  hybrid  automata  and  SlSf  differ  in  se¬ 
mantics:  a  hybrid  automaton  is  typically  defined  with 
may-semantics  with  respect  to  the  discrete  transitions, 
whereas  SlSf  employs  must-semantics.  In  other  words, 
a  transition  in  SlSf  is  taken  as  soon  as  the  transition 
guard  is  enabled  subject  to  some  numerical  aspects  with 
zero-crossing  detection,  whereas  the  hybrid  automaton 
still  has  the  freedom  to  stay  in  the  current  location  as 
long  as  the  location  invariant  has  not  been  violated.  In 
case  of  non-deterministic  hybrid  automata,  trajectory- 
equivalence  means  that  the  behaviors  of  the  original  hy¬ 
brid  automaton  will  be  exhaustively  explored.  Our  ap¬ 
proach  incorporates  additional  randomization  steps  into 
the  resulting  SlSf  diagram.  In  this  way,  in  every  run, 
the  diagram  produces  a  possibly  different  trace  that  still 
reflects  a  trajectory  from  the  original  hybrid  automaton 
semantics.  After  running  more  and  more  simulations,  we 
get  a  better  and  better  approximation  of  the  reachable 
state  space  of  the  original  hybrid  automaton. 


does  not  apply  to  this  class  of  models,  so  verification  is 
not  possible.  In  our  setting,  we  benefit  from  clear  and  un¬ 
ambiguous  hybrid  automata  semantics  and  may  formally 
verify  properties  of  the  hybrid  automata  prior  to  trans¬ 


lating  them  to  SlSf  diagrams.  Pajic  et  al.  25  33-35 


consider  a  similar  problem  of  converting  timed  automata 
encoded  in  Uppaal  27  to  SlSf  diagrams.  However, 
in  their  translation,  they  consider  only  runs  of  Uppaal 
models  that  obey  the  must- semantics.  In  our  work,  be¬ 
yond  considering  the  much  more  expressive  framework 
of  hybrid  automata  (as  timed  automata  are  a  subclass 
of  hybrid  automata) ,  we  provide  a  translation  handling 
the  non-determinism  by  producing  trajectory-equivalent 
SlSf  diagrams.  Operational  semantics  of  (purely  dis¬ 
crete)  Stateflow  have  been  developed  [24],  and  alterna¬ 
tive  formalizations  of  discrete  semantics  have  been  inves¬ 


tigated  using,  e.g.,  translation  from  Stateflow  to  C  38 


In  contrast  to  these  prior  works,  we  focus  on  continuous¬ 
time  Stateflow  diagrams.  Another  recent  line  of  research 
focusses  on  the  translation  from  Hybrid  Communicat¬ 
ing  Sequential  Processes  (HCSP)  to  Simulink  block  dia¬ 
grams  I6p3|44  .  In  our  work  we  consider  the  translation 
of  the  hybrid  automaton  model  which  is  extensively  used 
in  the  industry  for  CPS  modeling. 


Related  Work  Significant  research  has  been  done  on  the 
translation  of  SlSf  diagrams  into  other  analysis  tools, 
such  as  hybrid  systems  model  checkers  tummim 
29  -31  36  37  40,42  .  Agrawal  et  al.  [2]  suggest  an  al¬ 
gorithm  to  translate  SlSf  diagrams  into  the  equivalent 
HSIF  14  15]  36,  37  models.  The  Compositional  Inter¬ 
change  Format  (CIF)  provides  a  common  input  language 
focused  on  model  compositionality  for  networks  of  hy¬ 
brid  automata  |3].  Alur  et  al.  translated  SlSf  to  linear 
hybrid  automata  for  applying  symbolic  analysis  to  im¬ 
prove  test  coverage  of  SlSf  [4] .  In  a  different  setting, 
Schramnrel  et  al.  40  consider  the  translation  problem 


for  complex  SlSf  diagrams  where  involved  treatment  of 


zero-crossings  is  needed.  Manamcheri  et  al.  29  have  de¬ 


veloped  the  tool  HyLink  to  translate  a  restricted  class  of 


SlSf  to  hybrid  automata.  Minopoli  et  al.  30  31  have 


developed  a  theory  of  urgent  semantics  for  hybrid  au¬ 
tomata  and  the  SL2SX  tool  that  translates  a  restricted 
subset  of  SlSf  diagrams  to  hybrid  automata.  The  ap¬ 
plication  of  the  above  techniques  is  restricted  by  the  fact 
that  no  complete  semantics  of  SlSf  is  provided  (in  spite 
of  recent  progress  [8],  13, 23, 24[[29„38]). 

In  contrast  to  all  these  existing  works,  we  consider 
the  converse  direction,  i.e. ,  to  translate  a  given  hybrid 
automaton  into  an  SlSf  diagram.  Sanfelice  et  al.  [39] 
have  developed  the  hybrid  equations  toolbox  (HyEQ) 
to  approximately  simulate  the  hybrid  systems  that  may 
include  Zeno,  zero-crossing,  and  non-deterministic  be¬ 
haviors.  However,  the  applicability  of  the  Simulink  De¬ 
sign  Verifier  (SDV)  model  checkeiQintegrated  with  SlSf 


1  http : //www .mathworks . com/ product s/sldesignver if ier/ 


Contributions.  This  paper  has  four  primary  contribu¬ 
tions. 

(a)  This  is  the  first  work,  as  far  as  we  are  aware, 
to  provide  a  translation  scheme  from  hybrid  automata 
to  SlSf  diagrams,  which  is  useful  as  part  of  a  mod- 
el-based  design  (MBD)  process,  (b)  In  order  to  overcome 
the  difference  in  semantics  between  the  modeling  frame¬ 
works,  we  introduce  the  notion  of  trajectory-equivalence, 
and  show  how  the  conversion  preserves  trajectory-equiv¬ 
alence  with  respect  to  several  sources  of  non-determin¬ 
ism  in  hybrid  automata,  (c)  We  provide  an  implementa¬ 
tion  of  the  trajectory-equivalent  translation  scheme  as  a 
part  of  the  HyST  model  translation  framework  1 6  ,  which 
enables  completely  automatic  translation  of  existing  hy¬ 
brid  automaton  models,  (d)  We  show  the  applicability 
of  our  contributions  in  several  case  studies  where  hybrid 
automata  are  automatically  translated  to  SlSf  for  sim¬ 
ulation,  use  in  larger  SlSf  diagrams,  and  deployment  to 
actual  hardware.  For  one  case  study — a  closed-loop  buck 
converter — the  entire  correct-by-construction  MBD  pro¬ 
cess  is  illustrated,  from  verification  through  implemen¬ 
tation  in  hardware.  This  includes  formal  verification  of 
the  hybrid  automaton  in  SpaceEx,  translation  to  SlSf, 
code  generation  for  the  controller  in  SlSf,  then  subse¬ 
quent  compilation,  and  finally  execution  in  embedded 
hardware  controlling  the  physical  plant. 

Paper  Organization.  The  remainder  of  the  paper  is  orga¬ 
nized  as  follows.  After  introducing  the  necessary  back¬ 
ground  in  Sect.  [2]  we  present  our  trajectory-equivalent 
translation  scheme  in  Sect.  [3]  In  Sect.  [4]  we  evaluate  our 
approach  on  four  case  studies.  We  conclude  in  Sect.  [5] 
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2  Preliminaries 

In  this  section,  we  introduce  the  preliminaries  that  are 
needed  for  this  work.  We  first  define  a  hybrid  automaton 
model  and  discuss  its  semantics,  and  then  do  the  same 
for  SlSf  diagrams. 

2.1  Hybrid  Automata 

A  hybrid  automaton  is  formally  defined  as  follows. 
Definition  1  (Hybrid  Automaton).  A  hybrid  auto¬ 
maton  is  a  tuple  TL  =  ( Loc ,  Var,  Init,  Flow,  Trans,  Inv) 
with:  (a)  the  finite  set  of  locations  Loc,  (b)  the  set  of  con¬ 
tinuous  variables  Var  =  {x\, . . .  ,xn}  from  Rn,  (c)  the 
initial  condition,  given  by  Init(£ )  C  R”  for  each  loca¬ 
tion  £,  (d)  the  flow,  a  deterministic  function  Flow{£) 
from  the  variables  to  their  derivatives  for  each  location 
£,  (e)  the  discrete  transition  relation  Trans,  where  every 
transition  is  a  tuple  (£,g,v,£')  with:  (i)  the  source  loca¬ 
tion  £  and  the  target  location  i' ,  (ii)  the  guard,  given  by 
a  constraint  g ,  (Hi)  the  update,  given  by  a  mapping  v 
that  modifies  the  variable  valuation,  and  (f)  the  invari¬ 
ant  Inv(£)  C  Rn  for  each  location  l. 

We  use  the  common  .  (dot)  notation  to  specifically  indi¬ 
cate  components  of  TL  as  necessary,  e.g.,  TL.  Var  are  the 
variables  of  TL. 

The  semantics  of  a  hybrid  automaton  TL  is  defined 
in  terms  of  trajectories  as  follows.  A  state  of  TL  is  a  pair 
(£,  x)  that  consists  of  a  location  t  £  Loc  and  a  point  x  £ 
R™ .  Formally,  x  is  a  valuation  of  the  continuous  variables 
in  Var.  For  the  following  definitions,  let  T  =  [0,  A]  be 
an  interval  for  some  A  >  0. 

Definition  2.  A  trajectory  of  TL  from  state  s  =  (£,  x)  to 
state  s'  =  {£' ,  x')  is  a  pair  p  =  ( L ,  X),  where  L  :  T  — >  Loc 
and  X  :  T  — >  Rn  are  functions  that  define  for  each  time 
point  in  T  the  location  and  the  values  of  the  continuous 
variables,  respectively.  A  sequence  of  time  points  where 
location  switches  happen  in  p  is  denoted  by  ( Ti)i=o...k  € 
Tk+1.  In  this  case,  we  define  the  length  of  p  as  |r|  =  k. 
Trajectories  p  =  ( L ,  X),  and  the  corresponding  sequence 
(Ti)i-o...k,  must  satisfy  the  following  conditions: 

(a)  To  =  0,  Tj  <  ri+i,  and  =  A  -  the  sequence  of 
switching  points  increases,  starts  with  0  and  ends 
with  A, 

(b)  L(0)  =  £,  X(0)  =  x,  L(A)  =  l’,  X(A)  =  x'  -  the 
trajectory  starts  in  s  =  {£,  x)  and  ends  in  s'  =  (£' ,  x'), 

(c) ViVt  £  [Ti,Ti+1)  :  L(t )  =  L(ji)  -  the  location  is  not 
changed  during  the  continuous  evolution, 

(d) \/i  Vi  €  [Ti,n+ i)  ■  (X(i),X(t))  G  Flow(L(Ti)) 

holds  and  thus  the  continuous  evolution  is  consistent 
with  the  differential  equations  of  the  corresponding 
location, 

(e) Vi  Vi  £  [Tj,Tj-|_i)  :  X(i)  £  Inv^Lfrij)  -  the  contin¬ 
uous  evolution  is  consistent  with  the  corresponding 
invariants,  and 


(f)\/i  <  k  3(L(Ti),g,v,L(Ti+1))  £  Trans  :  Xend(i )  €  g 
A  X(ri+i)  =  v(Xend(i))  A  Xend(i)  =  limT^T-_i  X(r) 
-  every  continuous  transition  is  followed  by  a  discrete 
one,  where  Xend,{i)  defines  the  values  of  continuous 
variables  immediately  before  the  discrete  transition 
at  the  time  moment  r,yi. 

A  state  s'  is  reachable  from  state  s  if  there  exists  a  tra¬ 
jectory  from  s  to  s' . 

A  symbolic  state  s  =  {£,  TV)  is  a  pair,  where  l  £  Loc 
and  TZ  is  a  convex  and  bounded  set  consisting  of  points 
x  £  Rn.  The  continuous  part  TZ  of  a  symbolic  state  is  also 
called  region.  The  symbolic  state  space  of  TL  is  called  the 
region  space.  The  initial  set  of  states  Sinit  of  TL  is  defined 
as  [)((£,  Init(£)).  The  reachable  state  space  Reach('H)  of 
TL  is  defined  as  the  set  of  symbolic  states  that  are  reach¬ 
able  from  some  initial  state  in  Sinit,  where  the  defini¬ 
tion  of  reachability  is  extended  accordingly  for  symbolic 
states.  We  refer  to  the  set  of  all  the  trajectories  of  TL 
starting  in  Sinn  by  Traj (TL).  A  safety  specification  P  is 
a  given  set  of  symbolic  states.  A  hybrid  automaton  TL 
satisfies  a  safety  specification  P  iff  Reach("H)  C  P.  We 
are  interested  in  ensuring  that  the  hybrid  automaton  is 
correct,  i.e.,  satisfies  P,  and  then  subsequently  trans¬ 
late  it  for  simulation,  integration,  and  implementation 
in  SlSf  as  discussed  in  the  next  sections. 

2.2  Continuous- Time  Stateflow  Diagrams 

Simulink  is  a  graphical  modeling  language  for  control 
systems,  plants,  and  software.  Stateflow  is  a  state-based 
graphical  modeling  language  integrated  within  Simulink. 
Continuous-time  Stateflow  diagrams  provide  methods  for 
modeling  hybrid  systems  that  consist  of  continuous  and 
discrete  states  and  behaviors.  In  this  section,  we  describe 
a  restricted  subclass  of  continuous-time  Stateflow  dia¬ 
grams  to  which  we  translate  a  hybrid  automaton.  In 
particular,  we  focus  only  on  continuous-time  Stateflow 
state  transition  diagrams  and  we  do  not  consider  mod¬ 
els  with  hierarchical  states. 

Roughly,  a  Stateflow  state  transition  diagram  may 
be  thought  of  as  an  extended  state  machine  with  vari¬ 
ables  of  various  types.  In  addition  to  states,  Stateflow 
diagrams  may  have  junctions  that  are  instantaneous.  A 
transition  between  states  may  occur  at  each  simulation 
time  step,  whereas  multiple  junction  transitions  may  oc¬ 
cur  in  a  single  simulation  time  step. 

A  continuous-time  Stateflow  diagram  (see  Fig.  [5|  is 
roughly  analogous  to  a  hybrid  automaton,  but  their  be¬ 
havior  differs  in  several  ways.  In  particular,  Stateflow  di¬ 
agrams  (1)  are  deterministic,  (2)  have  urgent  transitions 
with  priorities,  and  (3)  have  events  such  as  enabled  tran¬ 
sitions  that  are  determined  at  runtime  by  zero-crossing 
detection  algorithms. 

We  define  Stateflow  diagrams  more  formally  now. 
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Figure  2:  Snippet  of  a  general  continuous-time  Stateflow 
diagram  with  a  state  is,  a  junction  j,  and  four  transi¬ 
tions  7~i  —  T4. 

Definition  3  (Stateflow  diagram).  The  tuple  S  = 
(Locs,  Junes ,  Vars,  Transs ,  Actionss )  defines  the  State- 
flow  diagram.  Here,  (a,)  Locs  is  a  finite  set  of  states  (also 
known  as  locations ),  (b)  the  junctions  Junes  are  like 
locations,  but  all  of  which  may  be  evaluated  in  a  sin¬ 
gle  simulation  event  step  (i.e. ,  they  are  instantaneous 
“states”),  (c)  Vars  is  a  finite  set  of  variables  of  vari¬ 
ous  types,  and  for  our  formalization  we  assume  they  are 
real- valued,  (d)  the  Actionss(is)  for  each  location  is 
are  actions  described  by  Matlab  or  C  statements  that 
are  performed  at  different  event  times  subdivided  into 
entry,  during,  and  exit  actions,  where  the  entry  (resp. 
exit)  action  is  executed  only  once  when  entering  (resp. 
exiting)  the  state  and  the  during  action  performs  the 
continuous-time  evolution  of  the  variables  of  Vars  ac¬ 
cording  to  a  differential  equation  (this  happens  strictly 
between  entering  and  exiting) ,  (e)  the  discrete  transition 
relation  Transs  where  every  transition  r  £  Transs  is  for¬ 
mally  defined  as  atuple  (is,  Guards ,  Updates ,  TPs,i's ): 

(i)  the  source  location  or  junction  is  £  Locs  U  Junes 
and  the  target  location  or  junction  i!s  £  Locs  U  Junes, 

(ii)  the  guard,  given  by  a  constraint  Guards,  must  be 
satisfied  for  a  transition  to  be  taken,  (in)  the  update, 
given  by  a  mapping  Updates,  defines  which  variables  in 
Vars  are  modified,  and  to  what  value  (unmodified  vari¬ 
ables  keep  their  value),  and  (iv)  the  priority,  given  by 
TPs,  is  a  natural  number  between  1  and  od(is) — the 
outdegree  of  (number  of  transitions  leaving)  the  state  or 
junction  is — that  indicates  the  order  in  which  transi¬ 
tions  are  taken  if  more  than  one  is  enabled. 

Simulating  an  SlSf  diagram  produces  a  simulation 
trajectory,  which  is  closely  related  to  a  trajectory  of  a 
hybrid  automaton. 

Definition  4  (Simulation  trajectory).  For  an  initial 
state  Xq,  a  time  bound  Tma,x,  error  bound  6  >  0,  and  time 
step  r  >  0,  a  simulation  trajectory  (of  length  k)  is  a 
sequence  a  =  ((Ri,ti))i—  i...k,  where  Rq  =  {cco},  to  =  0, 
Ri  C  R",  ti  £  R-°,  and  (a)  V*  :  0  <  U+i  —  <  t, 

tk  =  7max,  (b)  \/i\/t  £  [ti,U+ 1]  :  the  simulation  state 
after  time  t  is  in  Ri,  and  (c)  Vi  :  dia(Ri)  <  5. 

Here  dia(-)  denotes  the  diameter  and  5  is  used  to 
bloat  the  simulation  trajectory  to  handle  numerical  er¬ 
rors;  picking  (5  =  0  represents  the  typical  result  of  a 


(idealized)  numerical  simulation  of  an  SlSf  diagram.  We 
note  that  the  various  actions  (e.g.,  entry,  during,  and 
exit  actions,  and  transition  updates)  are  evaluated  se¬ 
quentially,  while  hybrid  automaton  actions  are  executed 
concurrently.  By  Trac^iS)  we  denote  the  set  of  all  simu¬ 
lation  trajectories  of  an  SlSf  diagram  S  with  parameter 
6.  A  simulation  trajectory  a  satisfies  a  safety  specifica¬ 
tion  P  if  every  element  a.Ri  C  P,  i.e.,  P  contains  the 
states  of  the  simulation  trajectory  with  time  projected 
away.  An  SlSf  diagram  S  satisfies  a  safety  specifica¬ 
tion  P  if  all  simulation  trajectories  Trac,s(i5>)  satisfy  P. 
Note  that  in  practice,  any  simulation  trajectory  is  finite- 
length,  although  we  avoid  a  finite-length  assumption  in 
the  definition  of  simulation  trajectories  to  relate  possibly 
infinite  trajectories  of  a  hybrid  automaton  with  similar 
possibly  infinite  simulation  trajectories.  Moreover  note 
that  our  definition  of  a  trajectory  does  not  allow  instan¬ 
taneous  location  switches  in  the  hybrid  automaton.  This 
restriction  is  necessary  for  practical  purposes  because 
SlSf  requires  executing  a  (however  small)  simulation 
step  in  each  state. 

3  Translating  a  Hybrid  Automaton  to  a 
Continuous-Time  Stateflow  Diagram 

We  describe  our  main  contribution,  namely  how  to  trans¬ 
late  from  a  hybrid  automaton  to  an  SlSf  diagram.  For 
different  classes  of  hybrid  automata,  different  transla¬ 
tions  may  be  used,  and  we  discuss  two  classes  primarily 
based  on  whether  the  hybrid  automaton  is  deterministic 
or  not. 

To  compare  simulation  trajectories  of  an  SlSf  di¬ 
agram  with  trajectories  of  a  hybrid  automaton,  we  in¬ 
troduce  the  concept  of  correspondence.  Here  we  assume 
that  the  S  parameter  of  a  simulation  trajectory  is  equal 
to  zero. 

Definition  5  (Correspondence).  A  trajectory  p  of  a 
hybrid  automaton  TL  and  a  simulation  trajectory  a  (with 
5  =  0)  of  an  SlSf  diagram  S  correspond  to  each  other  if 
the  sequences  of  discrete  locations,  transitions,  and  tran¬ 
sition  times  encountered  in  both  are  the  same,  and  the 
continuous  points  of  the  trajectory  and  the  simulation 
trajectory  match. 

The  primary  goal  of  our  construction  is  to  ensure 
that  the  set  of  simulation  trajectories  Trac^S)  for  the 
SlSf  diagram  can  be  trajectory-equivalent  to  the  origi¬ 
nal  hybrid  automaton. 

Definition  6  (Trajectory-Equivalence).  An  SlSf 
diagram  S  is  trajectory-equivalent  to  a  hybrid  automa¬ 
ton  TL  if,  for  every  trajectory  p  of  R,  there  exists  a  cor¬ 
responding  (Definition  [5])  simulation  trajectory  a  of  S , 
and  for  every  simulation  trajectory  a  of  S,  there  exists 
a  corresponding  trajectory  p  of  TL. 
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3.1  Translating  different  classes  of  hybrid  automata 

As  already  outlined  in  Sect,  [l]  one  main  difference  be¬ 
tween  hybrid  automata  and  SlSf  diagrams  is  the  ab¬ 
sence  of  non- determinism  in  SlSf  diagrams.  There  are 
several  sources  of  non-determinism  in  the  general  hybrid 
automaton  formalism. 

1.  Transitions.  If  there  is  more  than  one  outgoing  tran¬ 
sition  in  a  location,  any  of  them  can  be  taken  as  long  as 
the  guard  is  enabled  and  the  target  location’s  invariant 
is  satisfied  after  applying  the  transition  update. 

2.  Dwell  times.  The  amount  of  time  that  a  hybrid  au¬ 
tomaton  remains  in  a  location  is  only  determined  by  the 
invariant  and  the  transition  guards  -  it  is  forced  to  leave 
the  location  only  by  the  invariant.  It  is  not  sufficient  for 
the  guard  to  be  enabled  at  some  point  in  time,  as  the 
automaton  can  still  choose  to  remain  in  the  location  un¬ 
til  the  invariant  becomes  false. 

3.  Initial  states.  A  hybrid  automaton  is  allowed  to  start 
in  a  whole  region,  which  may  be  an  uncountable  number 
of  possible  initial  states. 

4.  Updates.  Updates  in  transitions  may  be  non-deter- 
ministic.  This  gives  a  (possibly  uncountable)  number  of 
successor  states  after  a  discrete  transition. 

5.  Flows.  Flow  definitions  in  locations  may  be  uncer¬ 
tain.  We  do  not  consider  this  source  of  non-determinism 
in  this  paper. 

For  the  translations,  we  make  the  following  assump¬ 
tions  on  the  original  hybrid  automaton. 

Assumption  1  The  hybrid  automaton  TL  is  Zeno-free, 
which  means  that  only  finitely  many  discrete  transitions 
may  be  taken  in  finite  time. 

Translating  deterministic  hybrid  automata  is  fairly 
straightforward,  so  we  first  discuss  how  to  translate  de¬ 
terministic  hybrid  automata,  and  then  discuss  the  more 
complex  non-deterministic  scenario.  There  may  be  addi¬ 
tional  numerical  issues  with  SlSf  that  are  outside  the 
scope  of  this  work.  For  example,  the  integration  of  the 
differential  equations  in  SlSf  may  not  be  exact,  which 
may  cause  differences  in  observed  behavior.  In  practice, 
simulations  can  be  made  arbitrarily  accurate  by  reduc¬ 
ing  the  simulation  time  step  at  a  computational  cost. 

3.1.1  Translating  a  deterministic  hybrid  automaton 

The  next  definition  states  when  a  hybrid  automaton  is 
deterministic. 

Definition  7.  A  hybrid  automaton  TL  is  deterministic 
if,  for  any  initial  state  (I,Xq)  £  Sinu  for  any  point  xq  £ 
Initial),  there  is  one  unique  trajectory  p  starting  from 
(£,Xo).  Otherwise,  TL  is  non-deterministic. 

Syntactic  restrictions  may  be  enforced  on  a  hybrid  au¬ 
tomaton  to  ensure  it  is  deterministic.  For  example,  a 
sufficient  condition  for  a  hybrid  automaton  to  be  de¬ 
terministic  includes  all  of  the  following  being  satisfied: 


(1)  at  most  one  discrete  transition  is  enabled  simulta¬ 
neously,  (2)  a  discrete  transition  guard  is  enabled  when 
the  continuous  flow  exits  the  invariant,  and  (3)  no  state 
can  be  mapped  onto  two  different  states  by  the  transi¬ 
tion  updates  26  Lemma  2].  Note  that  requirement  (2) 
is  not  an  urgent  definition  of  semantics,  but  it  is  a  condi¬ 
tion  that  ensures  an  enabled  transition  is  forced  to  occur 
once  it  becomes  enabled,  so  it  is  in  essence  a  syntactic 
restriction  that  enforces  urgency. 


Under  such  assumptions  that  enforce  a  hybrid  au¬ 
tomaton  to  be  deterministic,  the  translation  from  the 
deterministic  hybrid  automaton  to  an  SlSf  diagram  is 
straightforward  and  proceeds  as  follows.  Let  S  =  ( Locs , 
Junes ,  Vars,  Transs,  Actionss )  be  the  SlSf  diagram. 
Instantiate  Locs  =  TL.Loc ,  Junes  =  0,  and  Var$  = 
TL.Var.  For  each  location  I  £  Loc  and  each  correspond¬ 
ing  location  Is  £  Locs ,  and  for  each  variable  v  £  Var 
and  the  corresponding  variable  vs  £  Vars,  we  set  the 
Actionss  (Is-,  vs)  during  action  for  vs  to  be  equal  to  the 
flow  Flow(I,  v)  for  variable  v,  and  do  not  instantiate  the 
entry  and  exit  actions.  For  continuous-time  Stateffow 
models,  the  during  action  is  used  to  specify  an  ordinary 
differential  equation  for  variables,  so  in  essence  this  just 
copies  the  flow  from  TL  to  S  for  each  location  and  each 
variable,  and  the  other  action  types  (entry  and  exit) 
are  unused. 


Finally,  we  instantiate  the  transitions  as  follows.  For 
each  location  I  £  Loc  and  corresponding  location  Is  £ 
Locs,  and  for  each  transition  (I,g,v,I')  £  Trans  with  a 
natural  number  i  indicating  the  iteration  count  over  the 
transitions,  we  instantiate  a  transition  7  £  Transs  as  the 
tuple  (Is,  Guards,  Updates,  TPs,I's),  where  "/.Is  =  I, 
7.  Guards  =  g,  7-  Updates  =  v,  TPs  =  i,  and  7 .1's  =  I' ■ 
Since  TL  is  deterministic,  the  choice  of  the  transition  pri¬ 
ority  TPs  is  unimportant  as  only  at  most  one  transition 
is  enabled  at  a  time,  so  it  is  in  essence  set  arbitrarily 
to  i  based  on  whatever  iteration  order  is  chosen.  Ad¬ 
ditionally,  the  restriction  on  guards  and  invariants  to 
ensure  determinism  means  the  invariant  translation  is 
naturally  handled  through  the  translation  of  the  guard 
as  described  above. 

There  are  some  additional  minor  syntactic  transla¬ 
tions  that  also  must  occur  which  we  discuss  briefly.  The 
first  is  due  to  the  fact  that  updates  in  SlSf  are  evalu¬ 
ated  sequentially,  whereas  in  a  hybrid  automaton  they 
are  evaluated  concurrently,  so  additional  temporary  vari¬ 
ables  are  introduced  to  handle  this  as  necessary  (e.g.,  the 
hybrid  automaton  update  x'  :=  x  +  1 A  y'  :=  x  is  rewrit¬ 
ten  to  the  SlSf  update  x'tmp  :=  x\x’  :=  Xtmp  +  1  ~,y'  := 
Xtmp,  where  Xtmp  is  a  fresh  temporary  variable). 

The  second  more  significant  difference  is  related  to 
how  SlSf  identifies  events  during  execution  or  simula¬ 
tion,  which  is  influenced  in  part  by  the  simulator  not  be 
infinitely  precise  and  have  numerical  errors.  In  particu¬ 
lar,  this  influences  event  detection  such  as  when  transi¬ 
tions  are  enabled  and  may  be  taken,  and  this  is  irnple- 
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merited  using  zero-crossing  detection  algorithms  inside 
the  simulation  routines  of  SlSf. 

In  particular,  if  a  guard  is  only  enabled  at  one  (sin¬ 
gular)  point  in  time,  it  will  almost  surely  not  be  de¬ 
tected  by  the  zero-crossing  mechanisms  used  by  SlSf, 
and  the  transition  is  usually  missed.  In  order  to  not  ex¬ 
clude  certain  behaviors  systematically,  we  consider  an  e- 
relaxation  of  each  guard  constraint,  similar  to  the  relax¬ 
ations  considered  in  translations  from  SlSf  to  hybrid  au¬ 
tomata  30  .  For  instance,  a  guard  constraint  of  the  form 


x  =  c A y  <  x  becomes  c  —  e  <  x  <  c  +  eAy  <  x  —  e.  The 
simulation  time  step  can  then  be  chosen  small  enough 
such  that,  based  on  the  value  of  e  and  the  Lipschitz  con¬ 
stant  of  the  dynamics,  no  transitions  will  be  missed. 

Although  this  may  permit  more  behaviors  than  the 
original  hybrid  automaton,  it  critically  prevents  transi¬ 
tions  from  being  missed,  which  is  necessary  for  trajec¬ 
tory-equivalence.  The  extra  behaviors  introduced  from 
this  necessary  step  can  be  reduced  by  considering  smaller 
values  of  e,  which  will  require  a  smaller  simulation  time 
step.  Reducing  the  time  step,  however,  will  be  at  the 
cost  of  additional  simulation  runtime. 

Example  Translation.  We  illustrate  the  translation  pro¬ 
cess  with  a  running  case  study  evaluated  in  more  de¬ 


tail  later  (Section  4.1).  A  deterministic  hybrid  automa¬ 
ton  for  this  example  appears  in  Figure  [3j  which  is  a 
model  of  a  closed-loop  control  system.  Specifically,  here  a 
periodically-updated  hysteresis  controller  is  used  to  reg¬ 
ulate  a  voltage  Vc  by  controlling  the  state  of  a  switch. 
This  is  a  flattened  (composed)  model  of  the  closed-loop 
system,  originally  consisting  of  a  timed  automaton  model 
of  the  hysteresis  controller  which  has  periodic  updates 
every  20  microseconds,  and  a  hybrid  automaton  model 
with  affine  dynamics  of  the  plant,  which  is  a  circuit 
known  as  a  buck  converter.  The  resulting  continuous¬ 
time  Stateflow  diagram  for  the  buck  converter  created 
using  our  translator  appears  in  Figure  [4]  (with  no  e- 
relaxations). 


3.1.2  Translating  a  non-deterministic  hybrid 
automaton 

For  a  non-deterministic  hybrid  automaton,  we  achieve 
trajectory-equivalence  by  replacing  non-determinism  in 
the  hybrid  automaton  by  (uniformly  distributed)  ran¬ 
dom  number  generation  in  the  SlSf  diagram.  In  this 
way,  by  executing  multiple  SlSf  simulations  we  can  ap¬ 
proximate  the  reachable  states  of  the  original  hybrid  au¬ 
tomaton. 

In  our  converter,  we  currently  support  initial  regions 
and  non-deterministic  updates  to  hyper-rectangles,  as 
well  as  deterministic  updates  which  can  be  arbitrary 
functions.  When  non-deterministic  assignments  or  initial 
states  are  used,  they  must  be  strict  subsets  of  the  invari¬ 
ant  of  the  target  or  initial  location,  respectively,  which 
we  note  can  be  statically  checked.  Under  this  assump¬ 
tion,  the  choice  of  the  initial  continuous  state  and  the 


non-determinism  possible  during  updates  can  be  done  by 
randomly  choosing  one  point  from  the  set  of  all  points 
available. 

In  the  rest  of  this  section,  we  focus  on  the  harder 
problem  of  non-determinism  from  the  transitions  and 
dwell  time.  We  first  give  an  overview  of  the  translation 
scheme.  Here  it  is  helpful  to  regard  the  trajectory  of 
a  hybrid  automaton  as  a  sequence  of  jumps,  and  after 
each  jump,  the  automaton  chooses  the  next  transition 
and  dwell  time.  The  crucial  difference  in  our  conversion 
is  that  the  choices  might  be  infeasible,  i.e. ,  violating  the 
invariant.  To  account  for  this,  we  incorporate  a  back¬ 
tracking  mechanism,  where  the  current  state  of  all  vari¬ 
ables  is  stored  when  entering  a  new  location.  Note  that 
time  is  an  entity  which  is  implicitly  present  in  all  hy¬ 
brid  automaton  models  and  we  can  always  add  a  (fresh) 
time  variable  t  with  flow  t  =  1.  This  allows  for  a  general 
translation  scheme  without  further  knowledge  about  the 
hybrid  automaton  under  consideration. 

We  translate  a  hybrid  automaton  location  £  into  a 
corresponding  location  cluster  £,  comprising  of  a  number 
of  SlSf  states,  junctions,  and  transitions.  The  clusters 
are  then  connected  by  the  same  transitions  as  in  the 
original  hybrid  automaton.  A  simulation  trajectory  of 
the  resulting  SlSf  diagram  then  visits  those  clusters. 
Inside  a  cluster,  the  execution  consists  of  three  phases, 
depicted  in  Fig.  [5] 

Three  phases  in  a  location  cluster.  In  the  first  phase,  we 
randomly  choose  a  transition  out  from  the  transitions 
currently  available.  In  the  second  phase,  we  choose  a 
time  threshold  T ■  In  the  final  phase,  we  incorporate  the 
original  continuous  dynamics  of  the  location  l. 

In  the  translated  model,  the  transition  tries  to  be 
taken  by  checking  the  original  guard  condition,  but  only 
after  dwelling  in  l  for  at  least  until  time  moment  T.  If 
the  transition  out  cannot  be  taken  possibly  due  to  an 
invariant  violation  -  in  the  time  frame  [T,  7(nax] ,  where 
Tjnax  is  the  maximum  simulation  time,  we  backtraci 
and  return  to  the  second  phase,  and  select  a  new  time 
threshold  T  which  is  strictly  less  than  the  previously- 
chosen  threshold.  To  ensure  termination,  we  bound  the 
number  of  times  backtracking  may  occur  before  trying 
T  =  0.  If  the  chosen  transition  can  still  not  be  taken,  we 
can  conclude  that  it  cannot  be  taken  at  all,  and  go  back 
to  the  first  phase,  this  time  trying  another  transition. 

3.2  Trajectory-Equivalence 

The  translation  process  described  above  maintains  the 
defined  notion  of  trajectory-equivalence.  For  this,  we 
consider  an  idealized  conversion,  where  there  are  no  nu¬ 
merical  errors  in  the  simulation,  the  value  of  e  is  zero, 

2  We  note  that  our  notion  of  backtracking  is  different  from  the 
one  that  occurs  with  multiple  junctions  in  SlSf.  In  particular, 
we  require  allowing  some  dwell  time  to  elapse  in  states,  whereas 
junctions  are  instantaneous. 
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Figure  3:  Composed  hybrid  automaton  model  of  the  closed-loop  feedback  control  system  for  the  buck  converter.  The 
buck  converter  plant  is  originally  modeled  as  a  hybrid  automaton  and  the  hysteresis  controller  is  modeled  as  a  timed 
automaton(see  Figure  [TTj) . 


[VC  <  V„f  -  Vtol  &&  U  >  T] 
{ td  =  0;  }  2 


[VC  <  Vref  -  Vtol  &&  U  >  T] 
{td  =  0;  }  i 


Figure  4:  Composed  SlSf  diagram  for  the  translated  closed-loop  feedback  control  system  for  the  buck  converter. 


transition  out  not  possible  in  [T,  7!nax] 


Figure  5:  High-level  location  cluster  translation  pattern 
consisting  of  three  phases.  The  location  cluster  £  denotes 
a  group  of  SlSf  states  and  junctions  which  reflects  the 
behavior  of  the  hybrid  automaton  in  the  location  £. 


and  the  SlSf  diagram  encodes  the  intended  semantics 
of  the  described  transformation  process. 


Theorem  1.  If  %  is  a  Zeno-free  hybrid  automaton  and 
S  is  the  SlSf  diagram  created  using  our  transformation 
process,  then  S  is  trajectory-equivalent  to  TL. 


The  proof  for  the  more  complex  non-deterministic  case 
is  given  in  the  Section  3.3.4  From  the  theorem  we  can 
conclude  that  our  translation  preserves  safety  properties. 


Corollary  1.  If  a  Zeno- free  hybrid  automaton  TL  satis¬ 
fies  a  safety  specification  P,  then  every  simulation  tra¬ 


jectory  of  the  translated  SlSf  diagram  S  also  satisfies 

P. 

3.3  Additional  Translation  Details  and  Proof 

3.3.1  Detailed  Translator  Description 

We  provide  a  detailed  description  of  our  translation.  It 
iteratively  converts  every  location  £  of  a  hybrid  automa¬ 
ton  and  its  outgoing  transitions  into  an  SlSf  diagram  of 
location  clusters  t  in  the  following  way  (see  Fig.  [61) .  We 
first  describe  the  data  structures  we  use  in  our  construc¬ 
tion.  The  list  outList  stores  the  ordering  in  which  the 
outgoing  transitions  of  the  location  £  are  considered  in 
the  simulation.  The  variable  out  keeps  track  of  the  cur¬ 
rently  chosen  outgoing  transition.  The  variable  Tv  stores 
the  first  time  moment  when  the  location  invariant  is  vio¬ 
lated.  7m ax  keeps  the  maximum  simulation  time,  i.e.,  the 
simulation  is  stopped  as  soon  as  this  bound  has  been 
reached.  The  variable  T  stores  the  time  threshold  af¬ 
ter  which  the  outgoing  transition  should  be  taken.  The 
variable  R  keeps  the  maximum  number  of  backtrackings 
we  want  to  allow,  whereas  r  stores  the  current  number 
of  backtrackings  in  the  location  cluster  £.  Finally,  the 
variable  t  stores  the  current  time  that  is  simulated.  In¬ 
troducing  this  variable  allows  us  to  model  going  back  in 
time  when  backtracking,  which  is  not  possible  for  the 
actual  simulation  time  that  is  tracked  by  SlSf. 
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We  continue  with  the  description  of  every  individual 
(SlSf)  state  in  our  construction.  The  current  simula¬ 
tion  time  and  the  hybrid  automaton  state  when  enter¬ 
ing  the  location  i  (and  respectively  the  location  cluster 
£)  is  stored  in  the  (SlSf)  state  £-ln.  Furthermore,  the 
algorithm  randomly  chooses  the  ordering  in  which  the 
outgoing  transitions  are  considered.  In  this  way,  we  han¬ 
dle  the  non-determinism  due  to  multiple  simultaneously 
enabled  transition  guards.  Finally,  the  variable  Tv  is  ini¬ 
tialized  to  Ttnax  as  we  do  not  have  any  information  about 
the  invariant  violation  at  that  moment. 

The  state  ^choose  covers  two  kinds  of  non-determinism. 
It  takes  care  of  the  situation  when  the  intersection  of  the 
invariant  and  the  transition  guard  is  non-singular,  i.e., 
when  a  switch  to  the  next  location  can  happen  not  only 
at  a  particular  time  moment,  but  within  a  time  interval. 
Note  that  if  the  continuous  dynamics  are  non-monotonic, 
there  can  be  multiple  disjoint  time  intervals  where  the 
guard  is  enabled.  We  resolve  such  situations  by  generat¬ 
ing  a  random  time  threshold  T  in  the  state  £choose  and 
allowing  the  discrete  transition  only  from  the  time  mo¬ 
ment  T  onward,  i.e.,  we  add  a  constraint  of  the  form 
t  >  T  as  a  part  of  the  transition  guard  for  every  outgo¬ 
ing  transition  from  the  location  t.  Thus,  we  disable  the 
SlSf  must-semantics  up  until  time  moment  T  to  mimic 
the  original  may-semantics  of  hybrid  automata. 

Note  that  we  also  use  the  state  ^choose  for  backtrack¬ 
ing  purposes.  We  observe  that  an  unfortunate  choice  of 
the  outgoing  transition  out  and  the  time  threshold  T  can 
lead  to  the  simulation  getting  stuck,  as  the  transition 
guard  of  out  is  not  enabled  in  the  time  frame  [T,  Tmax] 
and  thus  the  transition  cannot  be  taken.  In  such  cases, 
we  return  to  the  state  ^choose  to  select  a  further  time 
threshold  T.  For  this  purpose,  we  restore  the  simulation 
time  t  and  the  state  of  the  hybrid  automaton  from  the 
moment  we  entered  £  resp.  £.  Afterward,  we  can  choose 
the  next  time  threshold  from  the  interval  [t,  T}.  Here  we 
observe  that  in  general  before  reaching  the  time  thresh¬ 
old,  the  invariant  can  be  violated.  Thus,  we  actually  se¬ 
lect  a  new  threshold  from  the  interval  [t,  min(7_,  Tv)]-  In 
this  way,  we  end  up  with  a  sequence  of  monotonically 
decreasing  thresholds.  Still,  as  it  is  not  guaranteed  that 
the  chosen  threshold  is  eventually  equal  to  0,  we  add  a 
further  termination  criterion  by  bounding  the  number 
of  backtracking  by  some  user-defined  constant  R  >  0. 
The  last  time  before  exceeding  this  limit,  we  try  out  the 
weakest  threshold  T  =  0  to  ensure  that  we  have  covered 
all  cases.  If  the  transition  cannot  be  taken  at  all,  we 
either  proceed  with  a  further  outgoing  transition  (junc¬ 
tion  ji„)  or,  if  none  is  left,  the  simulation  is  stopped  and 
reports  an  actual  deadlock  in  the  model. 

The  continuous  evolution  corresponding  to  the  loca¬ 
tion  £  is  modeled  by  the  state  ^dweii-  We  can  leave  this 
state  under  two  conditions.  First,  the  invariant  can  be 
violated.  Then  we  store  the  time  moment  when  the  vio¬ 
lation  has  happened  in  the  variable  Tv  and  move  to  the 
state  ^choose  (via  junction  jv).  Note  that  if  we  have  al¬ 


ready  considered  all  the  outgoing  transitions  of  £,  we  will 
stop  the  simulation  since  a  deadlock  has  been  found.  In 
the  other  case,  the  time  threshold  T  can  be  reached.  We 
take  the  transition  to  the  successor  location  of  £  if  the 
guard  of  the  chosen  transition  out  is  enabled  and  after 
applying  the  update,  the  target  location’s  invariant  is 
satisfied  (junction  jt).  Furthermore,  here  we  also  check 
whether  the  maximum  simulation  time  Tma,x  has  been 
reached,  in  which  case  we  stop  the  simulation. 

In  the  following,  we  illustrate  the  translation  process 
using  an  example  simulation. 

3.3.2  Example 

We  consider  an  execution  in  some  location  cluster  for  a 
simple  location  £\  with  one  continuous  variable  x  and 
two  outgoing  transitions,  as  depicted  in  Fig.  [7]  For  sim¬ 
plicity,  assume  that  the  location  is  entered  at  time  t  =  0 
in  state  x  =  0  and  the  total  simulation  time  is  7jnax  =  20. 

First  we  store  the  current  continuous  state  (f,  x)  = 
(0,0).  Next,  in  phase  1,  we  choose  a  transition,  say,  the 
one  to  £2 ■  Then,  in  phase  2,  we  choose  a  random  mini¬ 
mum  dwell  time  in  the  range  [0,  20],  say  T  =  3.  The  sim¬ 
ulation  proceeds  in  phase  3  until  an  event  occurs.  In  this 
case,  events  are  either  violating  the  location  invariant 
x  <  10  or  enabling  the  guard  condition  of  the  selected 
transition  t  >  3  A  x  >  8.  The  guard  condition  is  enabled 
first,  at  state  ( t,x )  =  (4,8).  This  transition  cannot  be 
taken,  however,  as  the  target  invariant  would  be  violated 
after  applying  the  update  x  :=  2. The  simulation  contin¬ 
ues  until  the  next  event,  when  the  state  (f ,  x)  =  (5, 10)  is 
reached  and  a  violation  of  the  invariant  is  detected.  That 
is  why  the  simulation  goes  back  to  phase  2,  backtrack¬ 
ing  to  the  saved  state  ( t,x )  =  (0,0).  At  this  point,  it 
was  checked  that  for  all  T  >  3,  the  transition  cannot  be 
taken.  In  phase  2,  a  new  value  for  T  is  chosen  from  the 
restricted  interval  [0,3),  and  the  simulation  is  run  again 
in  phase  3.  After  reaching  the  same  conclusion  and  af¬ 
ter  further  backtracking,  a  finite  threshold  of  attempts 
is  reached,  and  T  =  0  is  forced.  Even  with  T  =  0  there 
will  be  a  violation  of  the  invariant  before  the  transition 
can  be  taken.  Then,  we  will  conclude  that  the  selected 
transition  can  never  be  taken  when  starting  in  the  state 
(t,  x)  =  (0,0).  Thus  we  can  safely  ignore  this  transition, 
go  back  to  phase  1  and  choose  the  transition  leading  to 
£ 3 ,  where  the  process  repeats. 

3.3.3  Translation  Correctness  and  Discussion 

Correctness.  The  proof  of  Theorem  [l]  required  three  as¬ 
sumptions,  mentioned  before  the  theorem  statement  and 
proven  below.  First,  we  assumed  the  simulations  were 
exactly  accurate.  Although  real  simulations  will  always 
have  some  error,  this  can  be  reduced  to  arbitrarily  small 
values  by  reducing  the  time  step  used  in  the  simula¬ 
tion.  Similarly,  for  the  second  assumption  we  can  con- 
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Figure  6:  General  location  cluster  of  some  location  t  with  n  outgoing  transitions,  (re-)  store_variables  stores  and 
restores  the  current  simulation  state  (including  the  time  variable  t)  from  when  entering  the  cluster,  respectively. 
permute(n)  returns  a  permuted  list  outList  with  all  integers  from  1  to  n.  pop (outList)  removes  and  returns  the  first 
element  from  outList.  chooseT  chooses  a  new  time  threshold  T.  A  subscript  “1”  indicates  that  a  transition  has  the 
highest  priority  among  all  the  outgoing  transitions  from  a  state/junction. 


Figure  7:  Snippet  of  an  example  hybrid  automaton  with 
three  locations  i\  —  £3. 

sider  smaller  and  smaller  values  of  e,  although  in  de¬ 
generate  cases  this  might  permit  extra  transitions  in 
the  simulation.  For  example,  a  degenerate  guard  like 
x  <  5  A  x  >  5  will  always  be  false,  but  any  positive 
e-relaxation  will  have  a  possible  transition  when  5  —  £  < 
x  <  5+£.  The  third  assumption  is  that  the  SlSf  diagram 
correctly  encodes  the  described  transformation  process. 
This  means  that  correctness  is  subject  to  possible  im¬ 
plementation  bugs  in  our  conversion  implementation  in 
HyST,  as  well  as  the  semantics  of  Stateflow.  In  addi¬ 
tion  to  the  trajectory-equivalence  theorem,  we  provide 
empirical  justification  for  the  correctness  of  the  imple¬ 
mentation  of  our  translation  scheme,  through  extensive 
case  studies  including  the  buck  converter  detailed  in  the 
main  body,  and  additional  case  studies  presented  later 
in  the  appendix. 

Non-determinism.  By  replacing  non-determinism  with 
random  number  generation,  some  behaviors  of  the  orig¬ 
inal  hybrid  automaton  might  be  obscured.  For  instance, 
a  non-deterministic  die  can  roll  a  six  forever,  while  the 
probability  of  this  behavior  for  a  random  die  approaches 
zero  as  more  rolls  are  taken.  We  always  deal  with  finite 
executions  in  a  simulation,  and  thus  end  up  with  a  finite 
number  of  choices,  so  there  is  still  a  nonzero  chance  that 
the  ‘right’  random  values  will  be  chosen,  assuming  that 
the  hybrid  automaton  is  Zeno-free. 


Generalizations.  Although  we  consider  a  large  class  of 
hybrid  automata,  further  generalizations  are  possible. 
For  example,  the  initial  sets  and  non-deterministic  resets 
in  our  framework  were  hyper-rectangles,  whereas  in  gen¬ 
eral  the  initial  state  could  be  in  a  non-convex  set,  and  the 
reset  might  be  an  arbitrary  function  which  maps  from  a 
single  state  to  a  non-convex  set.  To  handle  such  systems, 
we  need  a  way  to  sample  in  the  non-convex  destination 
sets,  which  may  be  possible  in  certain  situations,  but  is 
difficult  in  general.  One  possibility  would  be  to  require 
the  user  to  give  this  sampling  function. 

Another  generalization  possible  is  to  consider  non- 
deterministic  dynamics.  More  general  hybrid  automata 
may  include  differential  inclusions  or  other  non-deter¬ 
ministic  ways  for  the  continuous  states  to  evolve.  This 
could  be  handled  by  adding  ranged  inputs  to  the  sys¬ 
tem,  and  at  each  time  step  choosing  a  random  value  in 
the  range  for  each  input.  However,  as  the  time  steps  be¬ 
come  smaller,  the  random  inputs  will  approximate  the 
main  value  in  their  ranges,  which  in  practice  results  in 
poor  simulation  coverage.  An  alternative  is  to  choose  a 
time  step  where  the  inputs  will  vary,  such  that  a  trade¬ 
off  is  possible  between  the  amount  of  coverage  possi¬ 
ble,  and  the  effect  of  this  tendency  towards  the  mean. 
Other  simulation  methods,  perhaps  based  on  state  ex¬ 
ploration  mechanisms  such  as  rapidly-exploring  random 
trees  (RRTs)  [28]  may  also  be  possible. 

3.3.4  Proof 

Proof  (Theorem  Q]).  We  first  show  the  forward  direc¬ 
tion,  i.e. ,  given  an  arbitrary  trajectory  of  the  hybrid  au¬ 
tomaton,  there  exists  a  set  of  random  decisions  in  the 
constructed  SlSf  diagram  that  produce  a  correspond¬ 
ing  simulation  trajectory. 
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Recall  that  correspondence  (Definition  [5])  requires 
that  the  encountered  locations  can  be  the  same,  and  that 
the  deviation  in  continuous  states  can  be  bounded  by  an 
arbitrarily  small  constant. 

For  the  ordering  of  locations,  notice  that  the  ran¬ 
dom  choice  of  an  outgoing  transition  in  phase  1  of  the 
construction  can  pick  the  corresponding  transition  from 
the  trajectory.  Since  the  minimum  dwell  time  is  chosen 
randomly,  it  can  be  picked  to  be  arbitrarily  close  to  the 
dwell  time  in  the  hybrid  automaton  trajectory.  In  this 
way,  as  long  as  the  continuous  evolution  in  the  simula¬ 
tion  remains  close  to  the  hybrid  automaton  trajectory’s 
continuous  evolution,  every  transition  will  be  explored. 

The  second  part  of  correspondence  requires  that  the 
deviation  in  the  continuous  states  is  bounded.  We  show 
that  this  bound  can  be  chosen  to  be  arbitrarily  small 
across  both  every  continuous  evolution  and  after  every 
discrete  transition.  During  a  continuous  evolution,  if  the 
start  state  in  a  location  in  the  simulation  is  chosen  close 
to  the  start  state  in  the  corresponding  location  in  the 
hybrid  automaton  trajectory,  its  deviation  will  also  be 
bounded  as  a  function  of  the  Lipschitz  constant  (see 
Proposition  1  in  [20 1).  Thus,  for  a  single  bounded  con¬ 
tinuous  evolution  and  every  nonzero  final  state  deviation 
desired,  there  is  a  corresponding  nonzero  initial  state  de¬ 
viation  that  will  achieve  the  desired  closeness. 

During  initial  state  selection,  since  we  consider  hyper¬ 
rectangles,  the  set  of  states  is  bounded.  Randomly  choos¬ 
ing  states,  we  will  in  finite  time  pick  one  arbitrarily  close 
to  any  trajectory’s  start  state  in  the  hybrid  automaton. 

Finally,  for  updates,  the  dwell  time  of  a  simulation 
can  be  made  arbitrarily  close  to  a  hybrid  automaton 
trajectory,  and  since  the  state  can  be  made  arbitrarily 
close,  a  deterministic  update  function  (under  assump¬ 
tions  of  Lipschitz  continuity)  can  also  result  in  a  state 
arbitrarily  close  to  the  trajectory.  For  nondeterministic 
updates,  the  argument  is  similar  to  the  initial  state  se¬ 
lection,  and  thus  the  continuous  states  of  the  simulation 
remain  arbitrarily  close  to  the  hybrid  automaton  trajec¬ 
tory. 

The  sequence  of  discrete  transitions  between  the  tra¬ 
jectory  and  simulation  match.  Since  each  trajectory  is  a 
finite  sequence  of  discrete  transitions  (due  to  Zeno-free 
behavior)  and  continuous  evolutions  (each  of  which  can 
have  arbitrarily  small  error  between  the  trajectory  and  a 
possible  simulation),  the  accumulated  error  for  the  whole 
trajectory  can  also  be  made  arbitrarily  small.  Thus,  the 
constructed  SlSf  diagram  has  simulations  which  corre¬ 
spond  to  any  arbitrary  hybrid  automaton  trajectory. 

The  reverse  direction  in  the  proof  shows  that  any  ar¬ 
bitrary  simulation  has  a  corresponding  hybrid  automa¬ 
ton  trajectory.  Again,  we  proceed  by  decomposing  this 
into  showing  that  the  sequence  of  locations  is  the  same, 
and  that  the  deviation  in  the  continuous  state  is  bounded. 

Since  we  assumed  an  idealized  relaxation  where  e  is 
zero,  every  transition  in  the  simulation  exactly  matches 
the  guard  conditions  in  the  hybrid  automaton,  and  thus 


the  hybrid  automaton  can  match  the  simulation.  Every 
update  in  the  constructed  SlSf  diagram  is  also  copied 
from  the  automaton,  so  that  the  automaton’s  trajectory 
can  match  the  random  choices  made  by  a  simulation. 

For  continuous  trajectories,  the  simulation  will  choose 
some  dwell  time  where  the  invariant  remains  satisfied 
until  a  guard  becomes  true.  The  hybrid  automaton  can 
also  pick  the  same  dwell  time,  and  its  invariant  will  also 
remain  true  until  the  same  guard  condition  is  reached. 
Thus,  the  hybrid  automaton  can  pick  a  trajectory  which 
corresponds  to  the  simulation. 

Since  every  trajectory  of  the  hybrid  automaton  cor¬ 
responds  to  a  simulation  trajectory  of  the  SlSf  diagram, 
and  every  simulation  trajectory  corresponds  to  a  trajec¬ 
tory,  the  two  models  are  trajectory-equivalent.  □ 


4  Evaluation  and  Experimental  Results 


To  evaluate  the  translation  methodology  presented  in 
this  paper,  we  implemented  a  prototype  translator  that 
uses  the  HyST  intermediate  representation  for  source-to- 
source  transformation  of  hybrid  automata  1 6] ,  and  the 
SlSf  API  within  Matlab  (tested  with  versions  2014a 
through  2016a).  The  input  to  the  translator  is  a  hybrid 
automaton  TL  in  the  SpaceEx  XML  format.  Networks 
of  hybrid  automata  are  first  composed  within  HyST  to 
yield  a  single  hybrid  automaton  representing  the  net¬ 
work.  Once  parsed  in  the  tool,  an  object  representing 
the  syntactic  structure  of  TL  is  traversed,  and  then  the 
tool  applies  the  sequence  of  translation  steps  described 
in  Sect.  [3]  In  the  simulator,  we  varied  the  seeds  of  the 
uniform  pseudo-random  number  generator  rng  in  Mat- 
lab.  We  evaluated  the  prototype  tool  using  several  ex¬ 
amples.  For  this  we  first  computed  the  reachable  states 
of  the  models  in  SpaceEx  or  Flow* ,  then  performed  the 
translation  and  simulations  in  SlSf.  The  tool  and  ex¬ 
amples  are  available  for  download  [I]. 


Cl  Case  Study:  Buck  Converter  with  Periodic 
Hysteresis  Controller 


A  buck  converter  is  a  DC-to-DC  switched-mode  power 
supply  that  takes  a  DC  input  source  voltage  and  lowers 
(“bucks”)  it  to  a  smaller  DC  output  voltage  32  .  A  stan¬ 
dard  model  of  the  converter  has  three  modes,  where:  the 
switch  is  closed  and  the  voltage  source  is  connected,  the 
switch  is  open  and  the  voltage  source  is  disconnected, 
and  based  on  the  possible  dynamics  of  the  converter, 
a  third  mode,  known  as  the  discontinuous  conduction 
mode  (DCM),  where  the  current  is  not  allowed  to  go 
below  zero  (which  is  physically  unrealizable,  but  may 
occur  without  this  third  mode).  Interested  readers  may 
find  detailed  derivations  of  models  in  power  electronics 
textbooks  [4l] .  A  hybrid  automaton  model  of  the  closed- 
loop  buck  converter  (plant  and  timed  controller)  appears 
in  Fig.  [3] 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

62 


Stanley  Bak  et  al.:  Hybrid  automata:  from  verification  to  implementation 


11 


A  standard  closed-loop  controller  for  the  buck  con¬ 
verter  is  a  hysteresis  controller,  which  changes  the  mode 
of  the  buck  converter  plant  based  on  the  measured  out¬ 
put  voltage.  Its  operation  depends  on  opening  and  clos¬ 
ing  the  MOSFET  switch.  Intuitively,  it  operates  like  a 
thermostat,  i.e.,  the  switch  is  toggled  so  that  the  source 
voltage  is  connected  to  the  circuit  if  the  output  voltage 
is  too  low,  and  it  is  toggled  in  case  if  the  output  voltage 
is  too  high  to  disconnect  the  voltage  source.  We  note 


that  by  Kirchhoff’s  voltage  law  (KVL),  Vc  =  Vout  41 


In  part  to  avoid  switching  too  frequently,  a  hysteresis 
band  is  typically  used  so  switches  occur  when  Vout  > 
Vref  +  Vtoi  or  Vout  <  Vref  -  Vtol.  This  creates  a  volt¬ 
age  ripple  on  the  output  voltage  that  should  be  within 
a  given  range  VriP  of  the  desired  reference  output  volt¬ 
age  Vref.  Together,  these  define  a  safety  specification: 
P(t)  =  t>ts=>  Vout(t)  =  Vref  ±  VriP,  which  projected 
onto  the  phase  space  is  P  =  Vref  —  Vnp  <  Vout  < 
Vref  +  V-i.p  •  SpaceEx  is  used  to  verify  P  by  computing 
the  reachable  states  Reach("H)  (to  a  fixed-point)  from  a 
startup  state  where  the  initial  states  Smit  are  II  =  0  and 
Vc  =  0.  For  every  time  t  >  ts  after  a  startup  trajectory 
of  duiation  ts ,  if  Vref  VriP  V.  Vout  (f )  d  Vref  4  Vr%p ,  then 
the  converter  satisfies  the  specification  P. 


For  actual  implementations,  the  measured  voltage 
values  are  sensed  periodically  through  an  analog-to- 
digital  converter  (ADC),  and  subsequently,  the  control 
signals  are  sent  periodically  to  control  the  state  of  the 
buck  converter  transistor  (open/closed).  We  model  this 
periodic  update  process  as  a  timed  automaton  for  the 
controller  with  a  timer  variable  t ^  that  evolves  at  unit 
rate  and  is  upper  bounded  by  T  of  20  microseconds.  The 
reachable  states  of  the  closed-loop  buck  converter  hybrid 
automaton  are  computed  with  SpaceEx,  and  as  shown 
in  Fig.  [8j  the  model  satisfies  the  safety  specification  P 
for  a  sufficient  choice  of  VriP . 

A  hardware  setup  consisting  of  a  buck  converter  plant 
and  a  dSpace  DS1103  is  used  to  perform  the  experiments 
with  the  physical  buck  converter  plant.  The  DS1103  con¬ 
tains  a  Power  PC  processor  and  a  DSP  board  and  is 
used  for  implementation  of  the  hybrid  automata  in  both 
hardware-in-the-loop  (HiL)  simulations  with  a  “virtual 
plant”  (the  plant  model  simulated  on  the  DS1103  hard¬ 
ware)  and  the  actual  buck  converter  plant. 

The  hysteresis  controller  is  executed  on  the  DS1103. 
First,  we  generate  C  code  using  the  translated  SlSf  di¬ 
agram  in  Matlab,  then  compile  it  and  download  it  onto 
the  DS1103.  A  discrete  fixed-step  solver  with  a  time  step 
of  20  microseconds  is  used  for  the  code  generation  pro¬ 
cess  and  also  for  the  DS1103’s  sampling  and  control  pe¬ 
riods,  which  is  sufficiently  small  to  ensure  e  is  sufficiently 
small,  as  discussed  in  Sect.  [3]  The  measured  voltage  sig¬ 
nal  from  the  buck  converter  is  periodically  sensed  and 
sent  to  the  embedded  controller  through  an  ADC.  The 
embedded  controller  generates  Boolean  valued  signals 
and  these  are  converted  to  suitably  spaced  rectangular 


> 

b 

> 


Figure  8:  Reachable  states  of  the  hybrid  automaton 
computed  with  SpaceEx,  verifying  the  voltage-regulation 
property,  along  with  HiL  simulation  results  of  the  trans¬ 
lated  SlSf  diagram  on  the  DS1103  (“virtual  plant”), 
and  control  of  the  physical  plant  with  the  translated 
SlSf  diagram  (“actual  plant”).  Our  results  validate  the 
high-level  vision  of  correct-by-construction  control  im¬ 
plementation  from  Fig.  |TJ 

pulses  to  operate  the  MOSFET  switch  of  the  buck  con¬ 
verter  plant.  For  the  experiments  with  the  actual  plant, 
the  input  signals  fed  to  the  controller  (specifically  the  Vc 
voltage)  are  replaced  from  the  simulation  model  with  the 
measurement  of  the  actual  plant,  and  the  output  signals 
(the  desired  mode,  open  or  closed)  are  fed  to  the  actual 
plant  instead  of  the  simulation  model.  The  experimental 
results  are  recorded  and  a  comparison  to  SlSf  simula¬ 
tions  is  shown  in  Fig.  [8]  The  experimental  and  simu¬ 
lation  traces  are  contained  in  the  SpaceEx  reach  sets, 
which  validates  the  translation  correctness  (Theorem  [l]) 
and  that  the  safety  property  is  maintained  in  the  im¬ 
plementation  (Corollary  [l]).  Note  that  in  the  hardware 
experiments,  the  controller  has  essentially  been  deter- 
minized,  as  the  purpose  of  non-determinism  in  the  hy¬ 
brid  automaton  model  was  to  model  plant  inaccuracies. 


4.1.1  Additional  Details 


The  buck  converter  circuit  appears  in  Fig.|9(a)|  Parame¬ 
ter  values  used  for  the  case  study  appear  in  Figure [9(b)] 
A  hybrid  automata  network  model  of  the  buck  con¬ 
verter  plant  and  a  timed  automaton  of  the  hysteresis 


controller  appears  in  Fig.  11  where  9  is  a  synchroniza¬ 
tion  label  and  S  is  a  discrete  control  signal,  and  a  bisimi¬ 
lar  hybrid  automaton  model  after  flattening  (composing) 
the  network  was  shown  earlier  in  Fig.  [3]  The  composed 
model  from  Fig.  [3]  is  used  for  verification,  translation, 
and  code  generation  purposes  as  discussed  earlier,  while 
the  network  model  is  conceptually  simpler  and  illustrates 
the  decomposition  between  the  physical  plant  hardware 
and  the  controller.  The  physical  hardware  used  in  the 
evaluation  appears  in  Fig.  |10| 

Fig.[l3|shows  the  reachable  states  in  the  phase  space, 
and  illustrates  that  the  SLSF  simulations  are  contained 
in  the  reachable  states  computed  with  SpaceEx  and  gives 
empirical  evidence  for  the  correctness  of  the  translation. 
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Component  /  Parameter  Name 

Symbol 

Value 

Source  Input  Voltage 

VS 

24  V 

Desired  Output  (Reference)  Voltage 

Vre/ 

12  V 

Actual  Output  Voltage 

Vc  =  Vout 

12  V  ±  Vrip 

Hysteresis  Band  Tolerance 

Vtol 

0.1  V 

Voltage  Ripple  Tolerance 

Vrip 

0.6  V 

Load  Resistance  with  Parameter  Variation 

R 

10  ±  2%  n 

Capacitor  Value  with  Parameter  Variation 

c 

2.2  ±  2%  mF 

Inductor  Value  with  Parameter  Variation 

L 

2.65  ±  2%  mH 

Periodic  Updation  Parameter 

T 

20  /i  sec 

(b) 


Figure  9:  (a)  Buck  converter  circuit — a  DC  input  Vs  is  decreased  to  a  lower  DC  output  Vc  =  VQ  =  Vout.  (b)  Buck 
converter  parameter  values  and  variations. 
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Figure  10:  The  buck  converter  plant  controlled  with  a 
dSPACE  DS1103  system.  Our  results  controlling  the  ac¬ 
tual  plant  with  the  translated  controller  validate  the 
high-level  vision  of  correct-by-construction  control  im¬ 
plementation  from  Fig.  [T| 


f.2  Case  Study:  Yaw  Damper  Controller  for  Ifl 
Aircraft 


A  yaw  damper  is  modeled  as  a  multiple-input  multiple- 
output  (MIMO)  system  which  uses  the  aileron  and  rud¬ 
der  in  order  to  reduce  oscillations  in  the  yaw  and  roll 
angle  of  an  aircraft.  In  this  section,  we  use  the  proposed 
method  to  analyze  the  control  design  of  a  yaw  damper 
for  a  747  aircraft,  taken  from  the  Control  Systems  Tool¬ 
box  case  studies  in  Matlab. 

In  particular,  we  analyze  the  final  designed  controller, 
which  includes  a  washout  filter  capable  of  eliminating 
oscillations,  but  maintaining  the  spiral  mode.  The  spiral 
mode  is  a  desired  control  characteristic  in  yaw  damper 
systems,  where  an  impulse  input  from  the  aileron  will 
result  in  a  bank  angle  which  does  not  immediately  de¬ 
crease  to  zero. 

The  model  for  the  system  is  given  at  Mach  0.8  at 
40,000  ft  using  standard  linear  time-invariant  dynam¬ 


ics,  x  =  Ax  +  Bu.  There  are  four  physical  variables  in 
the  system  x  =  (x\,  X2,  £3,  2h)t,  which  are  sideslip  an¬ 
gle  ( xi ),  yaw  rate  (xf),  roll  rate  (X3),  and  bank  angle 
(£4),  represented  by  the  column  vector  x.  The  two  in¬ 
puts  u  =  (ui,U2)t ,  are  the  rudder  (rti)  and  aileron  (1x2). 
The  outputs  are  the  yaw  rate  and  bank  angle. 

The  specific  values  for  A  and  B  are: 


-0.0558 

-.9968  0.0802 

0.0415 

.00729 

0 

0.598 

-0.115  -0.0318 

0 

,B  = 

-0.475  0.00775 

-3.05 

0.388  -0.4650 

0 

0.153 

0.143 

0 

0.0805  1 

0 

0 

0 

This  physical  system  is  put  into  a  feedback  loop  with 
a  washout  filter,  which  has  a  single  variable  w  and  dy¬ 
namics  w  =  x 2  —  0.2  •  w.  The  filter  variable  is  combined 
with  the  yaw  to  produce  an  effect  on  the  rudder  input. 
In  particular,  the  washout  filter  adds  to  u\  the  value 
2.34  •  (12  -  0.2  -w). 

We  consider  analysis  of  a  system  model  which  has  the 
guarantees  given  by  a  real-time  scheduler,  which  periodi¬ 
cally  executes  the  washout  filter  and  sets  the  output  val¬ 
ues.  Between  controller  executions  we  take  the  output  of 
the  washout  filter  to  be  constant  (zero-order  hold).  The 
control  task  is  guaranteed  to  execute  every  period  using  a 
common  scheduler  like  Rate  Monotonic  (RM)  or  Earliest 
Deadline  First  (EDF).  There  is  non-determinism  in  the 
exact  time  the  controller  runs,  however,  due  to  the  offset 
of  the  execution  of  the  control  task  within  each  period. 
Since  the  control  logic  is  simple,  we  take  the  control  task 
to  be  nonpreemptive  and  short,  so  that  the  model  will 
sample  the  physical  system  and  update  the  filter  output 
at  a  single  point  in  time,  but  that  point  in  time  may  vary 
within  each  period.  Furthermore,  we  look  at  the  system 
response  due  to  an  impulse  input  from  the  aileron  from  a 
range  of  start  conditions.  We  take  the  initial  bank  angle 
to  be  between  0  and  0.1. 

This  system  was  modeled  in  SpaceEx,  and  reachabil¬ 
ity  analysis  was  attempted  in  both  SpaceEx  and  Flow*. 
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Figure  11:  Hybrid  automaton  model  of  the  buck  converter  plant  with  timed  automaton  of  the  hysteresis  controller 
as  a  network. 


Figure  12:  Left:  Buck  converter  Vc  versus  time,  with 
SpaceEx  reach  set  for  the  hybrid  automatom  model  in 
red,  and  black  points  from  10  simulation  traces  of  the 
translated  SlSf  diagram.  Right:  Detailed  and  zoomed 
view  illustrating  multiple  simulation  trajectories. 


Figure  13:  Left:  Buck  converter  Vc  versus  il  (phase 
space),  with  SpaceEx  reach  set  in  red,  and  black  points 
from  100  simulation  traces.  Right:  Detailed  and  zoomed 
view  illustrating  multiple  simulation  trajectories. 


where  they  occur  at  varying  offsets  from  the  start  of 
each  period. 

The  simulations  showed  the  expected  response  of  the 
system  when  using  a  controller  period  of  T  =  0.1.  The 
response  of  the  system  is  shown  in  Fig.  [14]  Here,  the  im¬ 
pulse  response  from  the  aileron  to  the  bank  angle  is  plot¬ 
ted,  which  does  not  immediately  converge  (spiral  mode), 
and  does  not  contain  excessive  oscillations.  Thus,  using 
the  technique  proposed  in  this  paper  we  are  able  to  an¬ 
alyze  a  system  which  cannot  be  directly  analyzed  using 
reachability  tools. 

This  system  can  be  analyzed  formally,  however  this 
requires  a  non-trivial  model  transformation  using  the 
technique  of  continuization,  as  well  as  using  a  smaller 
control  period.  Continuization  converts  the  periodically- 
actuated  model  into  a  continuous  one  with  bounded 
noise,  where  the  bound  is  based  on  the  controller  period 
and  maximum  rate  of  change  of  the  output  signal  7  . 
The  same  model  can  be  used  as  the  basis  for  the  con¬ 
version  using  continuization,  as  well  as  the  conversion 
to  SlSf  for  simulation  and  further  Matlab-based  anal¬ 
ysis  and  code  generation.  In  this  way,  the  conversion  to 
SlSf  is  one  part  of  a  larger  toolflow,  where  models  are 
first  created  in  SpaceEx,  possibly  converted  for  formal 
analysis  using  HyST,  and  then  can  be  directly  imported 
into  SlSf  after  the  conversion  described  in  this  paper  for 
simulation  and  controller  synthesis,  as  well  as  embedding 
in  a  larger  CPS  model. 


Due  to  the  large  number  of  discrete  switches,  however, 
neither  tool  is  able  to  directly  compute  reachability  (the 
computed  reach  sets  grow  exponentially). 

Instead,  we  investigate  the  system  using  our  conver¬ 
sion  to  SlSf  and  randomized  execution.  Since  the  main 
source  of  non-determinism  in  this  model  is  the  discrete 
switches,  we  can  investigate  simulations  of  the  system 


4-3  Case  Study:  Glycemic  Control  in  Diabetics 


Glycemic  control  is  an  approach  to  control  the  blood 
glucose  levels  in  insulin  dependent  diabetes  mellitus  pa¬ 
tients.  There  are  several  different  mathematical  models 
of  glycemic  control  used  to  design  insulin  infusion  de¬ 
vices  that  help  diabetic  patients  control  their  blood  glu¬ 
cose  levels  21  .  Here  we  investigate  a  nonlinear  hybrid 
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Figure  14:  50  simulations  of  the  yaw  damper  sys¬ 
tem.  Left:  The  spiral  mode  is  confirmed.  Right:  Non¬ 
determinism  in  controller  execution  time  causes  simu¬ 
lated  trajectories  to  cross. 


Figure  15:  100  simulations  of  the  glycemic  control  model 
with  simulations  and  reach  set  computed  by  Flow* 
(gray)  for  variable  G. 


system  of  the  glycemic  control  in  diabetic  patients  such 
that  all  dynamics  are  defined  by  polynomials.  The  math¬ 
ematical  model  is  described  by  the  following  ODEs: 

G  =  -0MG-X{G+GB)  +  g(t )  (1) 

X  =  -0.025V  +  0.000013/  (2) 

/  =  -0.093(7  + /B)  +  u(i)/12  (3) 

In  Equation  [l]  and  Equation  [3j  G  and  /  are  the  plasma 
glucose  concentration  and  the  plasma  insulin  concentra¬ 
tion  above  their  basal  value  GB  and  IBl  which  are  equal 
to  4.5  and  15,  respectively.  The  variable  X  shown  in 
Equation  [2]  is  the  insulin  concentration  in  an  intersti¬ 
tial  chamber.  Moreover,  g(t)  and  u(t)  are  the  influx  of 
glucose  and  the  insulin  control  input,  presented  in  Equa¬ 
tion  [4]  and  Equation  [5j  respectively. 

{i/60  iff  <30 

(120  —  i)/180  if  30<<<  120  (4) 

0  if  t  >  120 

{25/3  if  G(f)  <  4 

25/3(G(f)  —  3)  if  4  <  G(f)  <  8  (5) 

125/3  if  G(t)  >  8 

The  glycemic  control  was  first  modeled  in  SpaceEx  and 
then  translated  to  Flow*  by  using  the  HyST  model  con¬ 
verter.  This  model  is  nonlinear,  non-deterministic,  and 
includes  4  variables,  9  locations  and  18  discrete  transi¬ 
tions  in  total.  The  simulations  of  the  glycemic  control 
model  translated  to  SLSF  are  shown  in  Fig.  [15}  We  sim¬ 
ulated  the  translated  model  with  100  different  random¬ 
ized  executions.  All  simulation  traces  of  G  are  contained 
in  the  reach  set  computed  by  Flow* ,  which  validates  the 
translation. 


4-4  Case  Study:  Fischer  Mutual  Exclusion 

Fischer  mutual  exclusion  is  a  timed  distributed  algo¬ 
rithm  that  ensures  a  mutual  exclusion  safety  property, 


Xi  :=  0 


Figure  16:  Fischer’s  mutual  exclusion  algorithm  for  a 
process  with  identifier  i  £  {1, . . . ,  N}.  Here,  g  is  a  global 
variable  of  type  {_L,  1, . . .  ,N},  Xi  is  a  local  variable  of 
type  R,  and  both  A  and  B  are  constants  of  type  R. 


namely  that  at  most  one  process  in  a  network  of  N  pro¬ 
cesses  may  enter  a  critical  section  simultaneously.  An  au¬ 
tomaton  for  Fischer  appears  in  Fig.  [16]  Fischer  involves 
two  real  timing  parameters,  A  and  B ,  and  mutual  exclu¬ 


sion  is  ensured  iff  A  <  B.  Let  Loc  =  { rem ,  try ,  waits,  cs}. 
We  translated  a  network  of  two  automata  ( N  =  2)  from 
SpaceEx  to  SLSF.  In  one  instance,  we  ensured  A  < 
B  by  picking  A  =  5  and  B  =  70,  so  mutual  exclu¬ 
sion  was  maintained,  which  we  verified  in  SpaceEx  using 
the  PHAVer  scenario.  In  the  other  instance,  we  ensured 
A  >  B  by  picking  A  =  75  and  B  =  70,  and  mutual 
exclusion  was  not  maintained.  Consequently,  we  could 
not  verify  this  instance  using  SpaceEx’s  PHAVer  sce¬ 
nario  since  a  location  cs  ~  cs  was  reachable,  corre¬ 
sponding  to  the  case  where  both  processes  are  in  the 
critical  section.  We  conducted  K  =  1000  simulations 
with  maximum  time  T  =  1000s  of  the  translated  SLSF 
model  in  each  case.  In  Fig.  17  we  show  respectively  the 
property  satisfaction  and  violation  through  the  auto¬ 
matic  translation  from  SpaceEx  to  SLSF  by  plotting 
the  corresponding  locations  versus  time,  where  differ¬ 
ent  colors  correspond  to  different  simulations.  In  the 
safe  case  {A  <  B),  the  locations  reached  via  simulations 
all  maintained  the  mutual  exclusion  property  and  were 
Loc2  \  {cs  ~  cs,  try  ~  cs,  cs  ~  try}.  In  the  unsafe  case 
(A  >  B),  the  locations  reached  via  simulation  included 
every  location  (e.g.,  all  16  locations  of  the  permutations 
of  LocN  for  N  =  2)  and  violated  the  mutual  exclusion 
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Figure  17:  Locations  reached  for  1000  SlSf  simulations 
of  Fischer,  where  different  colors  indicate  different  tra¬ 
jectories.  Left:  safe  case  ( A  <  B).  Right:  unsafe  case 
(A  >  B). 


property.  These  results  give  further  empirical  evidence 
for  the  correctness  of  the  translation  procedure. 

4-5  Additional  Case  Studies 
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No. 

Name 

Type 

|  Var  | 

|Loc| 

|  Trans] 

tc 

ts 

1 

biology_l 

NLC 

7 

1 

0 

8.894 

20.912 

2 

biology  _2 

NLC 

9 

1 

0 

7.892 

12.939 

3 

bouncing_ball 

LC 

2 

1 

1 

8.149 

11.960 

4 

brusselator 

NLC 

2 

1 

0 

7.428 

10.650 

5 

buckling_column 

NLC 

2 

1 

0 

7.738 

11.056 

6 

coup  ledVander  Pol 

NLC 

4 

1 

0 

8.202 

11.746 

7 

E5 

NLC 

5 

1 

0 

8.230 

36.635 

8 

fischer_N2_flat_safe 

LH 

6 

16 

82 

20.158  54.145 

9 

fischer_N2_flat_unsafe 

LH 

6 

16 

82 

19.287  59.627 

10 

glycemic_control_l 

NLH 

5 

3 

4 

8.319 

15.385 

11 

glycemic_control_2 

NLH 

5 

3 

4 

8.301 

15.567 

12  glycemic_control_polyl 

NLH 

4 

9 

18 

10.528  23.938 

13  glycemic_controLpoly2 

NLH 

4 

6 

10 

9.237 

19.341 

14 

helicopter 

LC 

28 

1 

0 

10.096  14.897 

15 

Hires 

NLC 

9 

1 

0 

7.912 

9.001 

16 

jet_engine 

NLC 

2 

1 

0 

7.667 

11.816 

17 

lac_operon 

NLC 

2 

1 

0 

7.586 

13.257 

18 

lorentz 

NLC 

3 

1 

0 

7.739 

11.253 

19 

lotka_volterra 

NLC 

2 

1 

0 

7.740 

11.025 

20 

circuits_n2 

NLH 

3 

3 

2 

9.39 

13.895 

21 

circuits_n4 

NLH 

5 

3 

2 

8.506 

14.202 

22 

circuits_n6 

NLH 

7 

3 

2 

8.585 

15.113 

23 

circuits_n8 

NLH 

9 

3 

2 

8.624 

15.386 

24 

circuits_nlO 

NLH 

11 

3 

2 

8.752 

15.813 

25 

circuits_nl2 

NLH 

13 

3 

2 

9.604 

19.837 

26 

OREGO 

NLC 

4 

1 

0 

9.157 

11.111 

27 

randgen 

LH 

3 

3 

6 

9.056 

15.112 

28 

Rober 

NLC 

4 

1 

0 

8.266 

16.999 

29 

roessler 

NLC 

3 

1 

0 

9.144 

12.771 

30 

smalLcircuit 

NLC 

5 

1 

0 

10.265  13.660 

31 

spiking_neuron 

NLH 

2 

2 

2 

8.703 

13.559 

32 

spring_pendulum 

NC 

4 

1 

0 

9.861 

6.251 

33 

vanderpol 

NLC 

2 

1 

0 

8.119 

12.226 

Table  1:  Overview  of  the  benchmark  problems  successfully  translated  to  SLSF  by  using  the  method  in  this  paper. 
Column  Type  presents  different  classes  of  dynamics,  where  LC,  NLC,  LH,  and  NLH  are  abbreviations  for  linear 
continuous,  nonlinear  continuous,  linear  hybrid,  and  nonlinear  hybrid,  respectively.  Columns  |  Var |,  |Loc|,  and  |  Trans \ 
show  the  number  of  variables,  locations,  and  transitions,  respectively,  while  tc  and  ts  show  respectively  the  time  our 
tool  required  to  translate  the  model,  and  the  time  to  simulate  the  translated  SlSf  diagram  twice. 
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Abstract 

Analog-mixed  signal  (AMS)  circuits  are  widely  used  in  various  mission-critical  applications  neces¬ 
sitating  their  formal  verification  prior  to  implementation.  We  consider  modeling  two  AMS  circuits  as 
hybrid  automata,  particularly  a  charge  pump  phase-locked  loop  (CP-PLL)  and  a  full-wave  rectifier 
(FWR).  We  present  executable  models  for  the  benchmarks  in  SpaceEx  format,  perform  reachability 
analysis,  and  demonstrate  their  automatic  conversion  to  MathWorks  Simulink/Stateflow  (SLSF)  format 
using  the  HyST  tool.  Moreover,  as  a  next  step  towards  implementation,  we  present  the  VF1DL-AMS 
description  of  a  circuit  based  on  the  verified  model. 

Category:  academic  Difficulty:  medium 


1  Context  and  Origins 

Many  analog-mixed  signal  (AMS)  circuits  are  widely  used  in  various  mission  cicritical  ap¬ 
plications  and  require  formal  verification  prior  implementation.  Formal  verification  methods 
construct  a  mathematical  model  M.  with  precise  semantics,  provide  extensive  analysis  with  re¬ 
spect  to  some  correctness  requirement  V ,  and  verify  that  A4  \=V  [2].  This  can  be  ascertained 
through  reachability  analysis  [1],  As  an  example  of  circuitry  that  can  benefit  from  formal  ver¬ 
ification  prior  to  field  implementation  and  deployment,  we  provide  two  potential  benchmarks 
for  hybrid  verification  research  community,  i.e.,  charge  pump  phase-locked  loop  (CP-PLL),  and 
full- wave  rectifier  (FWR). 

CP-PLL  integrated  circuits  are  widely  used  in  modern  mobile,  radio,  and  wireless  com¬ 
munication  applications  to  synchronize  a  high-frequency  signal  with  a  low-frequency  reference 
signal.  In  [8] ,  the  authors  use  SpaceEx  model  checking  tool  [6]  to  verify  the  global  convergence 
with  respect  to  phase  and  frequency  lock  for  a  digital  PLL.  An  FWR  converts  an  AC  electric 
input  signal  to  a  DC  output  signal,  and  formal  verification  through  reachability  analysis  has 
been  reported  using  different  model  checking  tools  in  [5],  except  SpaceEx.  We  develop  hy¬ 
brid  automaton  models  of  CP-PLL  and  FWR,  and  used  SpaceEx  [6],  a  reachability  analysis 
tool,  to  compute  the  over-approximated  sets  of  reachable  states  1 .  This  a  classical  fixed  point 
computation  tool  that  operates  on  symbolic  states. 

We  also  use  HyST  (Hybrid  Source  Transformer)  [3]  to  automatically  convert  the  hybrid  au¬ 
tomaton  models  developed  in  SpaceEx  to  MathWorks  Simulink/Stateflow  (SLSF)  models  2 .  It 
is  a  source-to-source  translation  tool  that  takes  input  in  the  SpaceEx  model  format,  and  trans¬ 
lates  it  to  the  formats  of  HyCreate,Flow*,  dReach,  C2E2,  Passel  2.0,  and  HyComp.  Additional 
tool  support  is  being  added  from  time  to  time.  Verification  and  validation  research  community 
may  use  HyST  to  automatically  transform  the  hybrid  automaton  models  in  SpaceEx  format  to 

1The  tool  is  available  online  from  the  SpaceEx  website  at:  http://spaceex.imag.fr/. 

2  The  executable  models  are  included  on  the  ARCH  website  and  are  also  available  online  from  the  HyST 
website  at:  http://verivital.com/hyst/. 
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Figure  2.1:  Block  diagram  of  the  PLL  circuit  with  a  PI  controller. 


other  formats  and  perform  reachability  analysis  using  aforesaid  model  checking  tools.  Finally, 
we  present  VHDL-AMS  description  of  an  FWR. 


2  Hybrid  Automaton  Modeling  of  CP-PLL  and  FWR 

In  this  section,  we  present  the  hybrid  automaton  modeling  of  CP-PLL  and  FWR. 


2.1  CP-PLL  Modeling 

We  consider  a  third-order  CP-PLL  as  described  in  [1],  It  consists  of  a  reference  frequency 
signal  generator,  a  phase  frequency  detector  (PFD),  a  charge  pump,  a  proportional-integral 
(PI)  controller,  a  voltage-  controlled  oscillator  (VCO)  and  a  frequency  divider  as  shown  in 
Figure  2.1.  The  state  variables  are  defined  by  the  voltages  across  the  capacitors  C),  Cpi,  and 
CP3,  i.e.,  Vi,  vpi,  and  vp  respectively.  Two  more  state  variables  are  defined  by  the  dynamics  of 
VCO  and  reference  frequencies,  i.e.,  <£„  and  <)re/,  respectively.  CP-PLL  is  designed  such  that 
4>v  locks  on  to  <t>ref ,  that  may  constitute  the  property  of  CP-PLL  to  be  verified.  This  locking 
is  ensured  by  PFD  using  the  phase  difference  of  <f>ref  and  <f>v  to  generate  ’UP’  or  ’DN’  signal 
for  the  charge  pump. 

The  ODEs  from  the  CP-PLL  circuit  diagram  can  be  readily  formed  using  the  traditional 
circuit  analysis  techniques,  i.e.,  Kirchoff’s  voltage  law  (KVL)  and  Kirchoff’s  current  law  (KCL). 
We  apply  KCL  at  node  1  of  the  circuit  used  to  implemented  the  analog  PI  controller  shown  in 
Figure  2.1 

i%  =  iCi  (2-1) 

We  can  write  the  above  equation  in  terms  of  voltage  across  capacitor  C,  as 

Ci.Vi  =  h.  (2.2) 


Rearranging  the  above  equation,  we  obtain 


(2.3) 
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We  apply  KCL  at  node  2  of  the  the  circuit  used  to  implemented  the  analog  PI  controller  in 
Figure  2.1  to  get 

iP  =  icp  i  +  iRj,  2  +  iRp  3  (2.4) 

Replacing  the  current  terms  with  voltage  terms  in  right  hand  side  of  above  equation,  we  get 

■  _n  ,  ,  vp1  ,  (vPi  -  vp ) 

V  —  CpiVpi  +  — - 1 - — - •  (2-5) 

^p3 

Rearranging  the  above  equation  for  vpi,  we  get 

1 

Rp3 

Next,  we  may  apply  KCL  at  node  3  to  get 


CpiRp3  Cpl 


(2.6) 


vpi  = 


Vpl 


C; 


pi 


1 

Rp  2 


*cp3  = iRp 3 

Re-writing  the  above  equation  in  terms  of  voltages,  we  get 

Vpl  Vp 


Cp2>Vp  — 

Rearranging  the  above  equation  leads  to 

.  _  Vpi 


R 


■p3 


Cp3  Rp3  Cp3  Rp3 


(2.7) 


(2.8) 


(2.9) 


For  the  VCO,  the  output  phase  <j>v  is  the  integral  of  the  frequency  and  the  input  voltages,  i.e., 
Vi,  and  vp  [7].  We  also  include  the  frequency  division  factor  N  to  obtain  the  ODE  as 


_  Ki  Kp  2ir 

~  NVi+  NVp+  N^° 


(2.10) 


and 

4>ref  =  27T  fref-  (2.11) 

Here,  Kj:  and  Kp  are  the  voltage-to-frequency  gains  for  Vi  and  vp  respectively,  and  /o  is  the 
frequency  of  VCO.  These  ODEs  depict  the  continuous  dynamics  within  each  discrete  location. 
The  input  to  the  PI  controller,  i.e.,  [ii,ip]T,  is  generated  by  the  charge  pump  depending  upon 
the  relative  phase  of  <f>v  and  <j>ref.  This  phase  difference  is  measured  by  PFD,  which  generates 
an  ‘UP’  signal  if  <j>ref  leads  <j>v,  and  ‘DN’  signal  if  <j)v  leads  4>ref-  An  ’UP’  signal  will  charge  the 
capacitors,  hence  increasing  the  voltages  across  the  capacitors  of  the  proportional  and  integrator 
channel,  i.e.,  vp  and  i respectively,  leading  to  an  increased  VCO  frequency.  On  the  other  hand, 
a  ’DN’  signal  from  PFD  will  tend  the  charge  pump  to  produce  current  in  reverse  direction  to 
discharge  the  capacitors,  hence  reducing  the  voltages  in  the  PI  channel.  The  reduced  vp  and 
Vi  voltages  will  result  in  a  reduced  </>v  to  make  it  track  4>ref-  Depending  upon  the  status  of 
Up/Down  signals,  there  may  be  four  discrete  locations  (i.e.,  the  input  varies  for  each  discrete 
location)  as  follows: 

1  Both0  (i.e.  Both  OFF):  The  input  vector  is  given  by  [ii,ip]T  =  [0,0]T 

2  Upi  (i.e.  UP  ON):  The  input  vector  is  given  by  [ii,ip)T  =  [/“P,/“P]T 

3  Bothi  (i.e.  Both  ON):  The  input  vector  is  given  by  [i»,  ip]T  =  [l“p  +  lfn,  Ipp  +  Ipn] T 
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Figure  2.2:  Hybrid  automaton  model  for  CP-PLL  system. 


4  Dni  (i.e.  DN  ON):  The  input  vector  is  given  by  [ii,ip]T  =  [lfn,  IpU] T 

Accordingly,  using  the  above  ODEs  and  the  inputs  defined,  a  hybrid  automaton  is  shown  in 
Figure  2.2.  The  component  values  used  in  the  model  are  as  per  Table  1  of  [1],  Moreover,  the 
input  values  are:  =  10.1/rA,  lfn  =  —10.1  pA,  and  J”p  =  505 pA,IpU  =  —505 pA.  The  guard 

conditions  for  discrete  transitions  are  formed  depending  upon  <f>ref  and  <j>v.  As  discussed  earlier, 
the  PFD  output  depends  on  whether  </>re/  leads  or  lags  with  respect  to  </>„.  If  the  initial  discrete 
location  is  Bothg,  the  automaton  jumps  to  Upi  if  <j>ref  leads  as  (f>ref  =  2-7T,  otherwise  it  jumps 
to  Dn\  if  (f)v  leads  as  <fiv  =  2n.  There  is  a  design  requirement  to  introduce  a  time  delay,  td, 
required  to  switch  off  both  the  charge  pumps.  This  is  represented  by  the  location  Bothi.  Once 
the  lagging  signal  reaches  zero,  the  automaton  jumps  to  this  location  and,  once  t  =  td ,  the 
automaton  transitions  back  to  Botho. 


2.2  FWR  Modeling 

We  consider  an  FWR  as  described  in  [5].  It  is  basically  a  full- wave  diode  bridge,  that  consists 
of  two  diodes  D\  and  D2 ,  a  capacitor  C  and  the  load  resistor  R  as  shown  in  Figure  2.3.  An  AC 
input  signal  is  supplied  to  the  circuit  through  a  center-tapped  transformer.  For  the  modeling 
purpose,  and  without  the  lack  of  generality,  we  use  two  AC  sources  as  shown  in  Figure  2.3.  This 
circuit  converts  the  input  AC  voltage  Vin  to  a  DC  voltage  V0,  at  its  output  measured  across  R. 
We  may  need  to  verify  that  V0  is  stable  within  ±l%Vmax  for  the  steady-state  operation,  where 
Vmax  is  the  maximum  value  of  the  input  AC  signal. 

For  modeling  purposes,  we  consider  Rd  as  the  forward  resistance  of  each  diode.  Let  the 
current  through  Rdl  C,  and  R  be  iRd,  ic ,  and  in,  respectively.  The  input  sinusoidal  voltage 
be  Vin  =  VmaxSin(2irft)i  and  the  output  voltage  across  the  load  resistor  R  be  V0,  where,  Vrnax 
is  the  maximum  amplitude  of  the  sinusoidal  signal  and  /  is  its  frequency.  For  model  checking 
purposes,  we  use  SpaceEx  that  requires  hybrid  automaton  model  with  linear  dynamics,  so  we 
model  the  input  AC  signal  using  a  second-order  differential  equation  [5].  We  define  another 
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D1 


Figure  2.3:  Schematic  diagram  of  FWR. 


Figure  2.4:  Hybrid  automaton  model  for  FWR  system. 


state  variable  xq  and  model  the  AC  input  by  ODEs  defined  as 

xo  =  Vn 


(2.12) 


and 

Vin=-(2nf)2x0  (2.13) 

The  solution  of  above  system  is  Vln  =  Vmaxsin(2'K ft)  such  that  the  initial  conditions  are 

x0  =  ^m‘ix  and  Vin  =  0.  Next,  we  consider  the  FWR  circuit  dynamics  to  form  ODE  for  Va. 
2? tj 

The  circuit  dynamics  depend  upon  the  operation  of  diodes  D\  and  Di-  Accordingly,  we  may 
form  three  different  topological  instances,  i.e.,  D\  ON  and  D2  OFF,  D\  OFF  and  D2  ON,  and 
both  the  diodes  OFF  when  V)n  <  Va.  There  could  be  a  fourth  topological  instance,  i.e.,  both 
the  diodes  ON  at  the  same  time,  but  this  is  not  practical  due  to  the  nature  of  the  sinusoidal 
input.  Therefore,  we  may  consider  three  topologies  one  by  one  to  form  the  ODEs  and  start 
with  the  topology  with  D\  ON  and  D2  OFF.  The  invariants  for  this  topological  instance  are 
Vin  >  V0  A  — Vin  <  V0-  Applying  KCL  at  the  node  joining  C  and  R  in  Figure  2.3,  we  get 


iRd  =  ic  +  iR 


and  we  can  express  the  above  equation  in  terms  of  voltages  as 


Vin  ~  Vo 

Rd 


=  cvQ  + 


K 

R  ' 


(2.14) 


(2.15) 
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V(,  Volts 

(a)  Phase  difference  vs.  Integrator  voltage 


V|,  Volts 

(b)  Cap.  p  Voltage  vs.  Integrator  voltage 


Time,  Sec  x  i  q'4 
(c)  Phase  difference  vs.  time 


Figure  3.1:  SLSF  plots  for  PLL  showing  stable  limit  cycles  and  <pv  locking  onto  <pref  within  0.2 
mSec. 


Figure  3.2:  Comparison  of  SpaceEx  reach  sets  and  SLSF  trajectories  for  PLL. 


Rearranging  the  above  equation  provides 


Vir 


(  1 


V  =  _ V  . 

°  RdC  °\RdC  RC 


(2.16) 


By  the  same  token,  for  D i  OFF  and  Do  ON  with  invariants  Vln  <  V0  A  —  V)n  >  V„,  we  use  KCL 
at  the  same  node  in  Figure  2.3  to  get 


V„  =  - 


Vn 

RdC 


-Vo 


(i?dC  +  Rc)' 


(2.17) 


For  the  topology  when  both  D\  and  D2  are  OFF,  the  sinusoidal  input  signal  is  cut  off  from  the 
entire  circuit  and  the  load  voltage  is  only  provided  by  the  capacitor.  The  invariants  for  this 
topological  instance  are  V*n  <  Va  A  —Vin  <  V().  Therefore,  we  get 


V0  =  - 


RC 


(2.18) 


Accordingly,  the  hybrid  automaton  model  of  FWR  is  shown  in  Figure  2.4.  In  addition,  we 
consider  the  VHDL-AMS  description  of  FWR  in  Section  A,  where  the  circuit  is  externally 
supplied  by  Vin. 


3  SLSF  Simulations  and  Reachability  Analysis 

Formal  verification  of  CP-PLL  constitutes  verifying  its  frequency- locking  property,  i.e.,  whether 
4>v  locks  onto  4>ref.  For  this  purpose,  we  need  to  compute  the  phase  difference  between  <pv  and 
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Figure  3.3:  Comparison  of  SpaceEx  and  SLSF  for  the  output  voltage  of  FWR  in  the  steady 
state,  showing  the  simulation  trace  containment  within  overapproximated  sets  of  reachable 
states. 


<t>ref-  The  SLSF  plots  for  the  phase  difference  vs.  integrator  voltage,  voltage  of  capacitor  pi  vs. 
integrator  voltage  V{,  and  the  phase  difference  versus  time  are  shown  in  Figure  3.1.  The  first 
two  plots  depict  a  stable  limit  cycle  highlighting  stability  properties  of  CP-PLL.  In  the  third 
plot,  we  show  that  the  phase  difference  between  <j)ref  and  (f>v  reaches  zero  within  0.2  mSec., 
signifying  that  <fiv  locks  onto  (j>ref  within  such  time  intervals. 

We  also  analyze  the  hybrid  automaton  using  SpaceEx,  and  a  comparison  of  the  first  few 
iterations  for  SpaceEx  and  SLSF  is  shown  in  Figure  3.2.  We  show  that  SLSF  simulation 
traces,  and  the  over-approximated  sets  of  reachable  states  computed  using  SpaceEx,  match 
for  the  first  five  iterations.  CP-PLL  requires  thousands  of  cycles  to  lock,  hence  there  will  be 
thousands  of  discrete  transitions  for  the  switching  logic  resulting  inaccuracy  due  to  SpaceEx 
overapproximations  [1].  It  is  evident  from  comparing  the  first  five  iterations  in  Figure  3.2  that 
SLSF  simulation  traces  are  contained  within  the  over-approximated  sets  of  reachable  states. 
We  also  conclude  that  the  SLSF  traces  exhibit  stable  limit  cycles,  and  that  frequency  locking 
is  achieved  within  0.2  mSec. 

As  evident  from  this  benchmark,  the  performance  of  reachability  analysis  tools  is  not  satis¬ 
factory  due  to  the  high  number  of  discrete  transitions  (practically  being  in  order  of  thousands) . 
It  is  pertinent  to  highlight  that  in  [4],  the  authors  have  used  a  variant  of  continuization  [1] 
to  address  this  problem  for  the  design  of  a  yaw  damper  system  for  a  747  jet  aircraft.  Con¬ 
tinuization  is  a  process  whereby  the  abstraction  of  a  hybrid  system  having  large  number  of 
discrete  transitions  is  obtained  by  a  continuous  system  with  an  extra  non-deterministic  input. 
The  authors  use  HyST  to  automatically  transform  the  model  and  perform  reachability  analysis 
using  Flow*  and  SpaceEx  to  display  satisfactory  results  in  [4] .  A  similar  approach  can  be  used 
for  this  benchmark  so  as  to  perform  reachability  analysis  using  SpaceEx  and  Flow*. 

We  perform  the  reachability  analysis  using  SpaceEx  under  the  steady-state  conditions  for 
FWR,  i.e.,  Vmax  =  4V,  Va  (0)  =  4V",  and  /  =  50 Hz,  as  shown  in  Figure  3.3.  The  steady-state 
SLSF  time  traces  for  the  output  voltage  are  contained  within  the  over-approximated  sets  of 
reachable  states  computed  using  SpaceEx. 

During  conversion  from  SpaceEx  to  SLSF  using  HyST,  the  conversion  time  noted  for  CP- 
PLL  is  1.633077  seconds  and  that  for  FWR  is  1.936676  seconds.  We  used  MATLAB  Release 
2015a  on  a  Windows  7,  64  bit  operating  system  with  Intel  Core  i7-2600  CPU  at  3.40  GHz  and 
16  GB  RAM. 
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4  Key  Observations 

Hybrid  automaton  modeling  and  reachability  analysis  of  CP-PLL  using  traditional  model  check¬ 
ing  tools,  such  as  SpaceEx,  is  an  extensive  challenge.  This  is  due  to  the  reason  that  CP-PLL 
requires  thousand  of  cycles  to  lock,  resulting  in  thousand  of  discrete  transitions  in  the  switch¬ 
ing  logic.  Therefore,  the  SpaceEx  analysis  did  not  produce  accurate  reachability  results  if  the 
analysis  is  run  for  an  extended  duration  of  time.  This  requires  some  advanced  techniques, 
such  as  continuization  [1]  that  is  demostarted  in  [4]  using  HyST,  SpaceEx,  and  Flow*.  For 
FWR,  SpaceEx  produced  a  run-time  error  due  to  non-affine  dynamics  as  the  model  had  pure 
sinusoidal  time-dependent  signal  as  an  input.  Therefore,  we  have  modeled  the  sinusoidal  input 
signal  using  the  second-order  ODEs  to  successfully  compute  the  reachability  analysis  results. 

5  Benchmark  Outlook 

Overall,  these  verification  benchmarks  have  medium  difficulty  level,  and  can  serve  as  a  first  step 
towards  a  benchmark  library  to  evaluate  reachability  and  verification  methods  for  AMS  circuits. 
These  benchmarks  are  open  to  the  continuous  and  hybrid  systems  verification  community  to 
evaluate  their  methods  and  tools. 
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A  Appendix:  VHDL-AMS  Description  of  FWR 

As  discussed  in  Section  2,  the  FWR  circuit  behavior  depends  upon  the  state  of  the  diodes  being  ON  or  OFF 
due  to  the  input  sinusoidal  signal.  We  assume  that  this  signal  is  supplied  externally,  and  form  the  description 
as  per  Equation  2.16,  Equation  2.17,  and  Equation  2.18.  It  should  be  mentioned  that,  in  VHDL-AMS,  we  must 
minimize  the  use  of  the  division  operation.  VHDL-AMS  models  are  typically  comprised  of  two  sections,  i.e., 
an  entity  and  an  architecture.  Entity  describes  the  model  interface  to  the  outside  world,  whereas,  architecture 
describes  the  function  or  behavior  of  the  model.  A  VHDL-AMS  description  is  given  below: 

library  ieee; 

use  ieee . electrical_systems . all ; 
use  ieee .math_real . all ; 
entity  fwr  is 

port  (  terminal  input:  electrical; 

terminal  output :  electrical  ) ; 
end  entity  fwr; 


architecture  dot  of  fwr  is 

quantity  vin  across  input  to  electrical_ref ; 
quantity  vout  across  output  to  electrical_ref ; 
constant  r  :  real  :=  1000;  —  load  resistance 
constant  rd  :  real  :=  0.1;  —  diode  forward  resistance 
constant  cap  :  real  :=  0.001;  —  capacitance 
begin 

if  vin  >=  vout  and  -vin  <=  vout  use 

vin  ==  vout’ dot  *  r  *  rd  +  vout  +  vout  *  rd  /  r;  —  diode  D1  ON 
elseif  vin  <=  vout  and  -vin  >=  vout  use 

-  vin  ==  vout ’dot  *  r  *  rd  +  vout  +  vout  *  rd  /  r;  —  diode  D2  ON 
elseif  vin  <=  vout  and  -  vin  <=  vout  use 

vout  ==  -  vout’ dot  *  r  *  cap;  —  Both  OFF 
end  if ; 

end  architecture  dot ; 
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Abstract 

Various  mission-critical  applications  necessarily  require  a  transformer  in  switching  con¬ 
verters  to  obtain  DC  isolation  between  the  converters’  input  and  output.  Since  DC-DC 
converters  are  the  switching  devices,  these  are  modeled  as  hybrid  automata.  We  present 
hybrid  automaton  modeling  of  two  main  types  of  transformer  isolated  DC-DC  converters, 
namely,  flyback  and  forward  converters.  We  have  also  catered  the  non-determinism  for 
both.  We  use  HyST  (Hybrid  Source  Transformation)  tool  to  automatically  generate  the 
models  in  SpaceEx  format,  perform  reachability  analysis,  and  then  automatically  convert 
the  models  into  Mathworks  Simulink  Stateflow  (SLSF)  using  HyST.  Thus  we  demonstrate 
effectiveness  of  HyST  tool  in  the  model-based  design  process.  The  HyST  user  needs  not 
to  manually  construct  or  modify  the  models  thus  saving  significant  amount  of  time  and 
efforts. 

Category:  academic  Difficulty:  medium 


1  Context  and  Origins 

DC-DC  converters  are  the  power  electronics  devices  that  are  extensively  used  in  automotives, 
industrial,  and  defense  related  applications  and  their  mission-critical  nature  necessitates  for¬ 
mal  verification  prior  implementation.  Over  the  period,  there  has  been  a  drastic  rise  in  power 
electronics-related  safety  recalls  in  the  automotive  industry.  For  example,  the  main  cause  for 
recall  of  around  700,000  Toyota  Prius  cars  in  2014  was  attributed  to  an  error  in  the  interaction 
between  a  boost  converter  and  its  software  controller  [11].  Likewise,  more  than  100,000  Toyota 
Prius  cars  were  recalled  due  to  an  inverter  failure  [12].  Therefore,  this  mission-critical  domain 
would  require  significant  confidence  in  the  modeling  accuracy.  This  can  be  ensured  through 
reachability  analysis  [1,6,7].  We  present  two  potential  benchmarks  related  to  transformer- 
isolated  DC-DC  converters  for  hybrid  verification  research  community.  Transformer  isolation  is 
implemented  by  introducing  a  transformer  at  the  converter  input.  In  addition  to  the  electrical 
isolation  between  the  input  and  the  output,  transformer-isolated  DC-DC  converters  have  some 
other  advantages  compared  to  their  non-isolated  counterparts  such  as  high  efficiency  and  low 
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manufacturing  cost  [4],  Due  to  their  advantages,  these  are  preferred  for  the  DC-DC  applica¬ 
tions  in  industrial  and  defense-related  control/communication  systems  and  distributed  power 
networks.  This  work  is  based  on  hybrid  automaton  modeling  of  two  main  types  of  transformer- 
isolated  DC-DC  converters,  i.e.,  flyback  converter  and  forward  converter.  This  is  a  series  of 
benchmarks  [6-8]  that  are  being  developed  to  benefit  from  formal  verification  prior  to  field 
implementation  and  deployment. 

Flyback  converter  may  be  regarded  as  a  transformer-isolated  buck-boost  converter,  whereas, 
forward  converter  acts  as  a  transformer-isolated  buck  converter.  We  develop  hybrid  automaton 
models  of  flyback  and  forward  converters,  and  use  SpaceEx  [5],  a  reachability  analysis  tool, 
to  compute  the  over-approximated  sets  of  reachable  states  1 .  This  is  a  classical  fixed  point 
computation  tool  that  operates  on  symbolic  states. 

We  also  use  HyST  (Hybrid  Source  Transformation)  tool  [2]  to  automatically  convert  the 
hybrid  automaton  models  developed  in  SpaceEx  to  MathWorks  Simulink/Stateflow  (SLSF) 
models  2.  It  is  a  source-to-source  translation  tool  that  takes  input  in  the  SpaceEx  model 
format,  and  translates  it  to  the  formats  of  HyCreate,  Flow*,  dReach,  C2E2,  Passel  2.0,  and 
HyComp.  In  addition,  it  is  also  used  to  automatically  generate  the  hybrid  automaton  models  in 
SpaceEx  format  as  per  user-defined  parameters  and  settings.  Additional  tool  support  is  being 
added  from  time  to  time.  Verification  and  validation  research  community  may  use  HyST  to 
automatically  transform  the  hybrid  automaton  models  in  SpaceEx  format  to  other  formats  and 
perform  reachability  analysis  using  aforesaid  model  checking  tools. 


2  Hybrid  Automaton  Modeling  of  Transformer-Isolated 
DC-DC  Converters 


We  present  the  hybrid  automaton  modeling  of  flyback  and  forward  converters  in  this  section. 
We  assume  that  transformer  losses  are  negligible  with  perfect  coupling  among  the  windings. 
The  transformer  is  modeled  using  a  parallel  magnetizing  inductance  Lm  at  the  input  side,  called 
the  primary  side.  The  winding  towards  the  output  is  called  the  secondary  winding.  Let  n  be  the 
turns  ratio  of  primary  to  secondary  windings.  Let  V\  and  V2  be  the  voltage  across  primary  and 
secondary  windings,  i\  and  %2  be  the  respective  currents,  and  let  n\  and  n2  be  the  respective 
number  of  turns.  Following  relations  hold  for  an  ideal  transformer 


U]_  _  V2_ 
n  i  n2  ’ 


(2.1) 


and 


n\i\  =  n2i  2- 


(2.2) 
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Figure  2.1:  Schematic  diagram  of  the  flyback  converter. 
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Figure  2.2:  Hybrid  automaton  model  for  flyback  converter. 


2.1  Flyback  Converter  Modeling 


We  consider  the  flyback  converter  in  open-loop  configuration  as  shown  in  Figure  2.1  exported 
from  PLECS  software  [9],  a  power  electronics  circuit  simulator.  The  switching  is  realized  by 
the  MOSFET  switch  and  the  diode  D\.  The  state  variables  are  defined  by  the  voltage  across 
the  capacitor  vq,  and  current  through  the  magnetizing  inductor  inductor  iLm-  The  MOSFET 
switch  is  operated  by  a  pulse  generator  of  constant  duty  cycle  D,  over  the  switching  time  period 
T.  The  operation  of  this  circuit  is  dependent  upon  the  state  of  the  MOSFET  switch,  i.e. ,  being 
ON  and  OFF,  resulting  into  two  modes: 


1.  Mode  1:  In  this  mode,  the  MOSFET  switch  is  ON  during  the  switching  cycle  0  <  t  <  DT , 
wherein,  the  input  DC  voltage  is  connected  to  the  primary  of  the  transformer.  This 
induces  the  current  in  the  secondary  winding  in  opposite  polarity  to  reverse  bias  the 
diode  (setting  it  to  OFF  state).  In  this  mode,  the  primary  of  the  transformer  is  charged, 
wheres,  the  diode  acts  as  an  open  switch  causing  the  capacitor  to  discharge  through  the 
load  resistance.  We  model  the  MOSFET  switching  loss  by  a  series  resistor  rsw.  The 
ordinary  differential  equations  (ODEs)  for  and  vc  for  this  mode  are  formed  using 
conventional  Kirchoff’  voltage  law  (KVL)  and  Kirchoff’s  current  law  (KCL).  Applying 
KVL  on  the  left  loop  gives 


dihm  _  T sw  ■  Vin 

T7  T  lLm  i  7  5 

l-'m 

whereas,  applying  KVL  on  the  loop  containing  R  and  C  gives 


(2.3) 


dvc  1 
hf  =  RCVC' 


(2.4) 


1The  tool  is  available  online  from  the  SpaceEx  website  at:  http://spaceex.imag.fr/. 

2  The  executable  models  are  included  on  the  ARCH  website  and  are  also  available  online  from  the  HyST 
website  at:  http://verivital.com/hyst/. 
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The  state  space  matrices,  during  the  switching  cycle  0  <  t  <  DT ,  are  thus  given  by 


r  SVJ 

Lm 

0 

,B±  = 

1 

Lm 

,X  = 

i'Lm 

0 

1 

RC 

0 

Vc 

(2.5) 


2.  Mode  2:  In  this  mode,  the  MOSFET  switch  is  OFF  during  the  switching  cycle  DT  < 
t  <  T,  thus  the  input  DC  power  supply  is  disconnected  from  the  primary  of  the  trans¬ 
former.  The  current  in  the  secondary  flows  in  upward  direction  hence  diode  is  forward 
biased  (in  ON  state).  We  first  consider  the  primary  winding  loop  and  apply  KVL.  Using 
Equation  2.1,  the  voltage  across  the  primary  is  given  by 


vi  =  -nvc, 


(2.6) 


such  that  the  negative  sign  is  due  to  its  opposite  direction.  Applying  KVL  in  the  primary 
winding  loop,  we  obtain  following  relation  for  the  magnetizing  inductor  current 


dl  Lm 
dt 


(2.7) 


The  current  through  primary  winding  is  the  same  as  current  through  Lm.  From  Equa¬ 
tion  2.2,  the  current  through  the  secondary  winding  is  given  by 


*2  =  niLm. 


(2.8) 


Consider  the  node  joining  R  and  C .  The  current  entering  this  node  is  i2.  Applying  KCL 
on  this  node,  we  get 


dvc  n  .  1 

dt  =  C%Lm  ~  RCV°' 


(2.9) 


The  corresponding  state  space  matrices,  during  the  switching  cycle  DT  <t<T:  are  thus 
given  by 


^2 


0 


n 


C 


n 


1 

RC 


0 

0 


(2.10) 


We  have  formulated  a  hybrid  automaton  model  of  flyback  converter  using  the  above  ODEs  as 
shown  in  Figure  2.2.  The  component  values  used  in  the  model  are  mentioned  in  Figure  2.1,  and 
adopted  from  [9]. 


2.2  Forward  Converter  Modeling 

The  forward  converter  may  be  regarded  as  a  transformer-isolated  buck  converter,  as  illustrated 
in  Figure  2.3  sketched  using  PLECS  [9].  It  has  a  MOSFET  switch,  and  three  diodes  D\,  D2, 
and  D%  to  realize  the  switching  operation.  We  consider  three  state  variables,  i.e,  magnetizing 
current  *im,  inductor  current  il,  and  capacitor  voltage  vc ■  Let  n\,  n2,  and  77.3  be  the  number 
of  turns  in  three  windings  of  the  transformer.  The  switching  modes  depend  on  the  state  of  the 
MOSFET  switch  as  well  as  the  fact  that  whether  inductor  current  <  0  and  the  magnetizing 
current  <  0.  This  results  in  six  different  modes  as  under. 
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Figure  2.3:  Schematic  diagram  of  forward  converter. 


1.  Mode  1:  In  this  mode,  the  MOSFET  switch  is  ON  during  the  switching  cycle  0  <  t  <  DT , 
wherein,  the  input  DC  voltage  is  connected  to  the  primary  winding  of  the  transformer. 
This  causes  D2  to  become  forward  biased  (ON),  and  D3  and  D3  to  become  reverse  biased 
(OFF).  Applying  KVL  to  left  most  loop  results  in 


dlLm 

dt 


Kr 

Lrr 


(2.11) 


whereas,  the  voltage  across  D3  is  ^  V)n .  Applying  KVL  to  the  loop  containing  L  and  C, 
results 


d%L  _  n3 
dt  niL 


(2.12) 


Consider  the  node  common  to  L,  C ,  and  R.  Applying  KCL  here  results 


dvc 

dt 


C 


IL 


RC 


Vc- 


(2.13) 


The  corresponding  state  space  matrices,  during  the  switching  cycle  0  <  t  <  DT ,  are  thus 
given  by 


0 

0 

0 

1 

Lm 

'i'Lm 

0 

0 

1 

L 

ii 

n  3 
n\L 

,X  = 

i-L 

,  U  —  Vin  • 

(2.14) 

0 

1 

c 

1 

RC. 

0 

Vc  _ 

2.  Mode  2:  The  MOSFET  switch  is  OFF  during  the  switching  cycle  DT  <  t  <  (1  —  D)T 
such  that  Vin  is  disconnected  from  the  primary  winding,  and  both  >  0  and  il  >  0. 
The  diodes  D\  and  D3  are  ON,  whereas,  D2  is  OFF.  The  input  voltage  is  applied  to  the 
winding  2  of  the  transformer  such  that  the  voltage  across  Lm  is  —Vin^.  This  results  in 
decrease  of  such  that 

di’Lm  Ul Vin  ^2 

dt  n,2Lm 

Since  L  discharges  through  the  load  resistor,  D3  remains  ON,  such  that  is  not  available 
to  charge  the  inductor  L.  This  gives  us 


diL 

dt 


dvc 

dt 


C 


IL 


RC 


vc- 


(2.16) 
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Figure  2.4:  Hybrid  automaton  model  for  forward  converter. 


The  corresponding  state  space  matrices  are 


0 

0 

0 

n  i 

Ti2Lm 

0 

0 

1 

L 

,B2  = 

0 

0 

1 

1 

0 

a 

RC  J 

(2.17) 


3.  Mode  3:  The  MOSFET  switch  is  still  OFF  during  the  switching  cycle  DT  <  t  <  (1  —  D)T 
such  that  iLm  <  0  and  %l>  0.  As  iLm  <  0,  diode  D\  becomes  OFF.  Overall,  the  MOSFET 
switch  and  diodes  D\  and  D2  are  OFF.  We  can  form  another  set  of  ODEs  as 


dlLm  _  „  dlL 
dt  ’  dt 


dvg 

dt 


C 


IL 


RC 


Vc- 


The  corresponding  state  space  matrices  are 


0 

0 

0 

o" 

II 

CO 

0 

0 

1 

L 

,B3  = 

0 

0 

1 

c 

1 

RC. 

0 

(2.18) 


(2.19) 


4.  Mode  4:  The  MOSFET  switch  is  OFF  during  the  switching  cycle  DT  <  t  <  (1  —  D)T 
such  that  both  and  iLm  <  0  and  il  <  0.  Following  set  of  ODEs  can  be  formed 


diLm  _  „  diL  dvc  _  jy 

dt  1  dt  ’  dt  RC  C 
The  corresponding  state  space  matrices  are 
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II 
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II 
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1 

RC. 
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(2.20) 


(2.21) 
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5.  Mode  5:  The  MOSFET  switch  is  OFF  during  the  switching  cycle  DT  <  t  <  (1  —  D)T. 
There  is  another  possibility  that  II  approaches  zero  while  iLm  is  still  non-zero,  thus  we 
have  another  condition  icm  >  0  and  ic  <  0.  This  gives  us 


diLm  _  niVin  diL_  _  dvg  _ _ 1_ 

dt  n2Lm  1  dt  1  dt  RC  C 

The  corresponding  state  space  matrices  are 


0 

0 

0 

n  i 

n^Lm 

II 

lO 

0 

0 

0 

Jh  = 

0 

0 

0 

1 

RC. 

0 

(2.22) 


(2.23) 


6.  Error  Mode:  Inherently,  the  maximum  possible  duty  cycle  for  the  forward  converter  is 
D  <  0.5.  Accordingly,  we  have  added  the  error  mode  in  the  model  to  accommodate  any 
deadlocks  due  to  wrong  selection  of  parameters. 

Using  the  above  ODEs  and  modes,  the  hybrid  automaton  model  of  forward  converter  is 
formulated  and  shown  in  Figure  2.4.  The  component  values  used  in  the  model  are  mentioned 
in  Figure  2.3  and  adopted  from  [10]. 

2.3  Closed-loop  Forward  Converter 

We  have  also  modeled  the  forward  converter  in  closed-loop  configuration  and  typically  used  the 
hysteresis  control  methodology  as  outlined  in  [3].  In  this  control  methodology,  the  capacitor 
voltage  vc  is  allowed  to  vary  within  a  hysteresis  band.  The  hysteresis  band  is  formed  by  defining 
an  upper  switching  boundary,  Vref  +  A,  and  a  lower  switching  boundary,  Vref  —  A,  where  Vref 
is  the  desired  output  voltage,  and  A  is  the  tolerance  level.  The  state  space  description  of 
the  model  remains  the  same  as  discussed  in  Section  2.3  and  shown  in  Figure  2.2,  whereas 
the  guards  t  >  DT  and  t  >  (1  —  D)T  are  changed  to  vc  >  Vref  +  A  and  vc  <  Vref  —  A, 
respectively.  Moreover,  the  invariants  t  <  DT  and  t  <  (1  —  D)T  are  changed  to  vc  <  Vref  +  A 
and  vc  >  Vref  —  A,  respectively. 


3  SLSF  Simulations  and  Reachability  Analysis 

We  have  automatically  generated  the  hybrid  automaton  models  in  SpaceEx  format  using  HyST 
tool  and  analyze  these  in  SpaceEx  environment.  Moreover,  we  have  automatically  translated 
the  same  SpaceEx  models  into  SLSF  format  using  HyST.  Formal  verification  of  the  flyback  and 
forward  converters  includes  verifying  the  corresponding  capacitor  voltage  and  inductor  current 
to  attain  a  stable  limit  cycle  in  settling  time.  For  the  flyback  converter,  we  require  that  vc  and 
i Cm  should  exhibit  a  stable  limit  within  settling  time  t$.  For  the  forward  converter,  we  require 
that  vc  and  should  exhibit  a  stable  limit  within  settling  time  ts- 

SpaceEx,  PLECS,  and  SLSF  results  for  the  capacitor  voltage  and  inductor  current  are  shown 
in  Figure  3.1.  It  is  evident  from  the  results  in  Figure  3.1  that  PLECS  and  SLSF  simulation 
traces  are  contained  within  the  over-approximated  sets  of  reachable  states.  We  also  conclude 
that  these  results  exhibit  stable  limit  cycle,  and  that  stable  voltage  is  attained  within  5  ms. 
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Figure  3.1:  Comparison  of  SpaceEx  reach  sets,  PLECS  and  SLSF  trajectories  for  the  flyback 
converter  showing  the  simulation  trace  containment  within  overapproximated  sets  of  reachable 
states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of 
capacitor  voltage  and  inductor  current. 
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Figure  3.2:  Comparison  of  SpaceEx  overapproximations  and  SLSF  trajectories  for  the  open-loop 
forward  converter,  showing  the  simulation  trace  containment  within  overapproximated  sets  of 
reachable  states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane 
plot  of  capacitor  voltage  and  inductor  current. 


We  perform  the  reachability  analysis  using  SpaceEx  for  forward  converter  as  shown  in  Fig¬ 
ure  3.2.  The  SLSF  time  traces  are  contained  within  the  over-approximated  sets  of  reachable 
states  computed  using  SpaceEx.  We  also  conclude  that  these  results  exhibit  a  stable  limit  cycle 
within  100  /is. 

There  are  various  sources  of  non-determinism  in  both  the  models  such  as  the  input  voltage 
( Vin ),  initialization  values  of  various  state  variables,  the  duty  cycle  of  the  PWM  signal  ( D ), 
and  the  time  period  of  PWM  signals  ( T ).  We  have  modeled  the  non-determinism  of  these 
parameters  for  both  types  of  converters. 

3.1  Reachability  Analysis  Results  -  Non-Determinism  in  Flyback 
Converter 

First  we  consider  the  non-determinism  in  for  the  flyback  converter,  such  that  it  is  allowed 
to  vary  from  11.9  —  12.1  V.  The  reachability  analysis  results  are  computed  using  SpaceEx  and 
shown  in  Figure  3.3.  We  consider  the  variations  in  initial  values  of  all  the  states  variables, 
i.e.,  iLm  and  vc ■  The  state  variable  iLm  is  initialized  for  a  range  of  0  —  0.5  A,  whereas  vc  is 
initialized  for  0  —  0.5  V.  The  reachability  analysis  results  are  computed  using  SpaceEx  and 
shown  in  Figure  3.4.  Next  we  consider  non-determinism  in  D ,  such  that  it  is  allowed  to  vary 
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Figure  3.3:  For  the  flyback  converter  model,  we  cater  the  non-determinism  for  the  input  voltage 
Vin  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time  (b) 
Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


Figure  3.4:  For  the  flyback  converter  model,  we  cater  the  non-determinism  in  initial  values  of 
i i,m  and  vc  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time 
(b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


from  0.449  —  0.501  s.  The  overapproximations  computed  using  SpaceEx  are  shown  in  Figure  3.5. 
In  the  last,  we  consider  the  variations  in  T  and  obtain  the  reachability  analysis  results  using 
SpaceEx  as  T  varies  between  19.96  —  20.04  fis,  as  shown  in  Figure  3.6. 


Figure  3.5:  For  the  flyback  converter  model,  we  cater  the  non-determinism  in  the  duty  cycle 
D  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time  (b) 
Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 
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Figure  3.6:  For  the  flyback  converter  model,  we  cater  the  non-determinism  in  the  sampling 
time  T  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time  (b) 
Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


Figure  3.7:  For  the  forward  converter  model,  we  cater  the  non-determinism  for  the  input 
voltage  Vin  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time 
(b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


3.2  Reachability  Analysis  Results  -  Non-Determinism  in  Forward 
Converter 

We  consider  the  non-determinism  in  Vln  for  the  forward  converter,  such  that  it  is  allowed  to 
vary  from  98  —  102  V.  The  reachability  analysis  results  are  computed  using  SpaceEx  and  shown 
in  Figure  3.7.  We  model  the  variations  in  initial  values  of  all  the  states  variables,  i.e.,  iLm,  *l> 
and  vc-  The  state  variables  iLm  and  ig  are  both  initialized  for  a  range  of  0  —  0.4  A,  and  vc 
is  initialized  for  0  —  0.4  V.  The  reachability  analysis  results  are  computed  using  SpaceEx  and 
shown  in  Figure  3.8.  Next  we  consider  non-determinism  in  D,  such  that  it  is  allowed  to  vary 
from  0.39  —  0.41  s.  The  overapproximations  computed  using  SpaceEx  are  shown  in  Figure  3.9. 
In  the  last,  we  consider  the  variations  in  T  and  obtain  the  reachability  analysis  results  using 
SpaceEx  as  T  varies  between  24.39  —  25.64  fj,s,  as  shown  in  Figure  3.10. 

3.3  Reachability  Analysis  Results  -  Closed-loop  Forward  Converter 

In  the  last  part,  we  present  the  reachability  analysis  results  for  the  closed-loop  forward  converter 
using  hysteresis  control  in  Figure  3.11.  For  the  hystersis-controlled  forward  converter  we  require 
that  II  and  vc  should  exhibit  a  stable  limit  cycle  within  the  settling  time  fg .  As  evident 
in  Figure  3.11,  both  ig  and  vq  exhibit  a  stable  limit  cycle  within  50  /is. 
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Figure  3.8:  For  the  forward  converter  model,  we  cater  the  non-determinism  in  initial  values 
of  and  vq  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current 

vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor 
current. 


Figure  3.9:  For  the  forward  converter  model,  we  cater  the  non-determinism  in  the  duty  cycle 
D  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time  (b) 
Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


Figure  3.10:  For  the  forward  converter  model,  we  cater  the  non-determinism  in  the  sampling 
time  T  and  overapproximations  are  computed  using  SpaceEx:  (a)  Inductor  current  vs  time  (b) 
Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 
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Figure  3.11:  Comparison  of  SpaceEx  and  SLSF  results  for  the  hysteresis-controlled  forward 
converter,  showing  the  simulation  trace  containment  within  overapproximated  sets  of  reachable 
states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of 
capacitor  voltage  and  inductor  current. 


4  Key  Observations 


Hybrid  automaton  modeling  and  reachability  analysis  of  transformer-isolated  flyback  converter 
has  medium  difficulty  level.  However,  modeling  and  analysis  of  forward  converter  is  more 
complex  with  three  state  variables  and  five  modes.  We  have  only  used  SpaceEx  to  perform 
the  reachability  analysis.  In  addition  other  reachability  analysis  tools  may  also  be  used  for  the 
reachability  analysis. 

We  have  not  considered  the  parasitics  in  modeling  of  transformer-isolated  DC-DC  converters 
that  will  further  increase  the  difficulty  level  of  this  benchmark. 


5  Benchmark  Outlook 


On  the  whole,  these  verification  benchmarks  can  serve  as  a  first  step  towards  a  benchmark 
library  to  evaluate  reachability  and  verification  methods  for  various  types  of  DC-DC  converters. 
These  benchmarks  are  open  to  the  continuous  and  hybrid  systems  verification  community  to 
evaluate  their  methods  and  tools. 
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Abstract — Formal  verification  requires  extensive  analysis  of 
a  given  mathematical  model  with  respect  to  some  correctness 
requirements  using  various  tools  and  techniques.  Manually  con¬ 
structing  models  of  a  given  device  in  various  formats  requires 
considerable  time  and  efforts.  Thus  we  automatically  generate 
the  hybrid  automaton  models  in  SpaceEx  format  using  HyST 
(Hybrid  Source  Transformer)  tool,  which  is  a  source-to-source 
transformation  and  translation  tool.  We  then  automatically  trans¬ 
late  these  SpaceEx  models  into  Mathworks  Simulink  Stateflow 
(SLSF)  for  analysis  thus  saving  significant  amount  of  time  and 
efforts.  We  present  various  power  electronics  circuits  benchmarks 
to  demonstrate  the  efficiency  and  effectiveness  of  HyST  in  model- 
based  design  process.  Safe  and  reliable  operation  of  these  circuits 
in  safety-critical  applications  necessitates  a  rigorous  modeling 
and  verification  process.  In  this  work,  we  use  SpaceEx  reachabil¬ 
ity  analysis  tool  for  formal  verification  of  such  circuits.  We  have 
used  this  computer-aided  modeling  technique  to  automatically 
generate  and  translate  the  models  and  verify  that  the  output  of 
a  given  model  remains  within  a  defined  stable  region  in  steady 
state. 

I.  Introduction 

Formal  verification  involves  constructing  a  mathematical 
model  A4  with  precise  semantics,  extensive  analysis  with 
respect  to  some  correctness  requirement  V,  and  verifying  that 
M.  |=  V  [1].  Reachability  analysis  has  been  used  for  formal 
verification  of  pre-defined  correctness  requirements  for  analog 
mixed  signal  circuits  [2].  In  this  work,  we  use  SpaceEx  [3], 
a  reachability  analysis  tool,  for  formal  verification  of  power 
electronics  circuits1.  Since  one  needs  to  build  the  model  of 
a  given  device  in  various  formats  so  as  to  perform  extenive 
analysis  using  various  tools  for  formal  verification.  Manually 
building  the  models  in  various  formats  requires  significant 
time  and  efforts.  Therefore,  we  have  used  a  new  tool  HyST 
(Hybrid  Source  Transformer)  [4]  to  automatically  generate  the 
hybrid  automaton  models  in  SpaceEx  compatible  format.  We 
also  use  HyST  to  automatically  convert  the  hybrid  automaton 
models  developed  in  SpaceEx  to  MathWorks  Simulink/State- 

The  tool  is  available  online  from  the  SpaceEx  website  at:  http://spaceex. 
imag.fr/. 


flow  (SLSF)  models  2 .  It  is  a  source-to-source  transformation 
and  translation  tool  that  takes  input  in  the  SpaceEx  model 
format,  and  translates  it  to  various  other  formats  such  as 
HyCreate,  Flow*,  dReach,  C2E2,  Passel  2.0,  and  HyComp. 
HyST  tool  is  being  updated  over  the  time  to  add  support  for 
other  analysis  tools.  The  verification  and  validation  research 
community  is  encouraged  to  use  HyST  as  this  computer- 
automated  analysis  saves  significant  time  and  efforts  in  model- 
based  design  process. 

Power  electronics  form  the  energy  middle-ware  and  used 
in  automobiles,  industrial  automation,  aerospace,  and  defense. 
Power  electronics  devices,  such  as  DC-DC  power  converters 
contain  switching  components  which  lead  to  discrete  behav¬ 
iors,  and  have  passive  components  that  exhibit  continuous 
dynamics  within  each  discrete  event.  Such  devices  can  be 
modeled  as  hybrid  automata  to  perform  reachability  analysis. 
A  signifcant  rise  in  the  safety  recalls  of  cars  manufactured  by 
automotive  industry  due  to  malfunction  of  power  electronics 
devices  has  been  reported.  As  an  exmaple,  about  700,000 
Toyota  Prius  cars  were  recalled  in  year  2014  due  to  an  error 
in  interaction  between  a  boost  converter  and  its  software 
controller  [5].  Later  in  year  2015,  more  than  100,000  Toyota 
Prius  cars  were  recalled  due  to  an  inverter  malfunction  [6]. 
Therefore,  such  mission-critical  devices  would  require  formal 
verification  prior  implementation. 

In  this  paper,  we  demonstrate  effectiveness  of  HyST  tool  in 
automatic  model-based  design  and  formal  verification  process 
using  four  case  studies  of  power  electronics  circuits.  First  two 
being  special  types  of  DC-DC  power  converters  called  center- 
tapped  Buck  and  boost  converters.  In  the  last  two  case  studies, 
we  use  two  improved  models  of  the  transformer-isolated 
DC-DC  power  converters  that  were  earlier  presented  in  [7], 
namely,  flyback  converter  (that  acts  as  a  Buck-boost  converter) 
and  forward  converter  (that  acts  as  a  Buck  converter).  This 
work  is  continuation  of  a  series  of  benchmarks  for  power 

-The  executable  models  are  available  online  from  the  HyST  website  at: 
http://verivital.com/hyst/. 
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Fig.  1.  Overview  of  the  HyST  conversion  process. 

electronics  circuits  [8]— [10]  that  are  being  developed  to  benefit 
from  formal  verification  prior  to  field  implementation  and 
deployment. 

II.  Automatic  Model  Generation  Using  HyST 

HyST  is  an  automatic  source-to-source  model  translation 
and  transformation  tool  that  takes  input  in  SpaceEx  format 
and  generates  models  in  SLSF,  HyCreate,  Flow*,  dReach, 
C2E2,  Passel,  and  HyComp  formats  [4].  The  support  for  other 
reachability  analysis  tools  will  be  added  from  time  to  time. 
HyST  can  be  beneficial  to  the  hybrid  systems  verification 
community  in  following  ways: 

1.  The  user  may  automatically  generate  a  model  file  for 
numerous  other  tools,  carry  out  the  analysis,  and  choose 
the  best  suitable  tool  for  the  system  under  consideration. 

2.  The  researcher  involved  in  development  of  hybrid  sys¬ 
tems  model  checkers  may  quickly  compare  the  perfor¬ 
mance  of  the  newly  developed  tool  with  other  tools. 

HyST  takes  input  in  SpaceEx  source  format,  parses  it  into  an 
intermediate  representation  (IR),  and  finally  prints  the  output 
source  in  a  format  specified  by  the  user.  This  conversion 
architecture  is  shown  in  Fig.  1.  IR  is  implemented  as  Java  data 
structures  to  encode  the  hybrid  automaton  model  components, 
whereas,  transformation  passes  may  be  regarded  as  the  model- 
to-model  conversions.  More  details  regarding  HyST  can  be 
found  in  [4]. 

In  this  paper,  we  use  HyST  as  a  benchmark  generator  for 
automatic  generation  of  hybrid  automata  models  in  SpaceEx 
format.  Thus  the  user  needs  not  to  manually  create  the  hybrid 
automata  models  through  SpaceEx  model  editor  saving  con¬ 
siderable  time  and  effort.  We  use  MATLAB’s  API  (application 
program  interface)  for  Java  that  enables  MATLAB  to  interact 
with  Java  programs  synchronously  or  asynchronously.  In  this 
automatic  model  generation  process,  we  need  to  instantiate  the 
model  components  per  Definition  2.1. 

Definition  2.1:  We  define  a  hybrid  automaton  model  by  a 
tuple  M  =  (i,  X,  Init,T,Inv,  F),  where: 

•  L  =  {7i ,  Z2,  is  a  finite  set  of  discrete  locations. 

•  X  is  a  finite  set  of  continuous  state  variables,  such  that 
V  x  £  A'  3  val{x)  £  R,  where  val(x)  is  a  vluation  of  x 
resulted  due  to  function  mapping. 

•  I nit  C  L0  x  X0  is  a  set  of  initial  conditions,  such  that 
Lq  C  L  and  Ao  C  X. 


Fig.  2.  Overview  of  automatic  model  generation  in  SpaceEx  format. 

•  T  =  (ls,le,g,r)  is  a  set  of  feasible  discrete  transitions 
allowed  among  the  discrete  locations,  where  the  corre¬ 
sponding  elements  of  the  tuple  are  the  start  location, 
end  location,  relevant  guard,  and  the  subsequent  reset, 
respectively. 

•  Inv  is  a  finite  set  of  invariants  for  each  discrete  location. 

•  F  is  a  set  of  ordinary  differential  equations  (ODEs)  that 
are  defined  for  each  location  l  £  L  over  the  continuous 
variables  x  £  X. 

We  implement  following  steps  (Fig.  2)  to  automatically  gen¬ 
erate  the  hybrid  automaton  model  using  MATLAB: 

1.  Instantiate  the  matrix/string  to  define  various  components 
of  the  hybrid  automaton  model  as  per  Definition  2.1. 

2.  Load  parameter  values  and  initialize  the  state  variables. 

3.  Call  the  parser  in  HyST  to  represent  these  components 
into  SpaceEx  data  structures. 

4.  Print  into  the  SpaceEx  model  format,  i.e.,  ’.cfg’,  and 
’.xml’  files. 

5.  Translate  and  print  the  model  into  the  SLSF  format. 

III.  Hybrid  Automaton  Model  Formulation 

The  power  electronics  devices  can  be  modeled  as  hybrid 
automata  as  these  exhibit  both  the  continuous  and  discrete 
behaviors  due  to  the  inherent  passive  elements  and  switches, 
respectivley  [11].  In  this  section,  we  discuss  the  modeling  of 
such  circuits  for  use  in  automatic  SpaceEx  model  generation 
process  and  translation  to  SLSF  format.  We  demonstrate  the 
effectiveness  of  HyST  tool  in  model-based  design  process 
using  four  different  types  of  power  electronics  circuits. 

For  the  model  formulation,  we  assume  the  transformer 
losses  to  be  negligible.  The  winding  at  the  input  is  called 
primary,  whereas  that  towards  the  output  is  called  secondary. 
The  dynamics  of  such  circuits  depends  on  the  operation  of  the 
MOSFET  switch,  i.e.,  being  ON  and  OFF.  We  consider  open 
loop  DC-DC  power  converters  such  that  the  MOSFET  switch 
is  operated  by  a  pulse  generator  of  constant  duty  cycle  D,  over 
the  switching  time  period  T.  The  state  variables  are  defined 
by  the  voltage  across  the  capacitor  vc,  and  current  through 
the  inductor  i l . 

A.  Center-Tapped  Buck  Converter  Model 

It  is  a  special  type  of  DC-DC  Buck  converter,  wherein,  the 
inductor  is  center-tapped,  i.e.,  a  contact  is  made  to  a  point 
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Fig.  3.  Schematic  diagram  of  center-tapped  Buck  converter. 


halfway  along  the  winding  of  an  inductor.  The  schematic  of 
the  converter  is  shown  in  Fig.  3.  Let  n  be  the  turns  ratio  of 
primary  to  secondary  windings,  n\  be  the  number  of  turns 
before  the  center-tap,  and  712  after  the  center-tap.  For  a  tapped 
inductor,  let  vl  be  the  overall  voltage  across  the  entire  number 
of  turns,  then 


vL  ni+n2  .  n\ 

n  —  —  =  - =  1-1 - . 

v2  n2  n2 


(1) 


The  state  of  the  MOSFET  switch,  i.e.,  being  ON  and  OFF, 
results  into  two  modes.  The  third  mode  results,  when  the 
MOSFET  switch  is  OFF  and  <  0. 

1.  Mode  1:  During  the  switching  cycle  0  <  t  <  DT, 
MOSFET  switch  is  ON  and  diode  is  OFF.  The  input  DC 
voltage  source  Vlrl  supplies  the  primary  of  the  inductor. 
In  this  mode,  the  entire  inductor  is  charged  and  diodes 
acts  as  an  open  switch  to  charge  the  capacitor  and 
supply  the  load  resistance.  The  ODEs  for  and  vc  may 
be  formulated  using  conventional  Kirchoff  voltage  law 
(KVL)  and  Kirchoff’s  current  law  (KCL).  We  use  KVL 
on  the  outer  loop  containing  L,  R,  and  C  that  results  in 


cUl 

dt 


(2) 


whereas,  applying  KCL  on  the  node  joining  L,  R,  and  C 
results 

dvC  1  .  1 

^t  =  cil~rcvc-  (3) 


The  state  space  matrices,  during  the  switching  cycle  0  < 
t  <  DT,  are  thus  given  by 


Ai  — 


0  -i 

jp 

c 


1_ 

L 

1 

RC 


,Bi  = 


,x  = 


IL 

Vc 


i  v  VL 


(4) 

2.  Mode  2:  In  this  mode,  the  MOSFET  switch  is  OFF 
during  the  switching  cycle  DT  <  t  <  T,  thus  Vln 
is  disconnected  from  the  primary  of  the  transformer. 
However,  the  current  in  the  secondary  (equivalent  to  ni l 
as  derived  from  (1))  still  flows  hence  the  diode  is  forward 
biased  (in  ON  state).  We  first  consider  the  secondary 
winding  loop,  apply  KVL  and  use  (1)  to  form  ODE 
as 

diL 


dt 


=  -yVC- 


(5) 


Applying  KCL  on  the  node  joining  L,  R,  and  C,  we 
obtain  following  ODE. 


dvg 

dt 


C 


IL 


RC 


VC- 


(6) 


Fig.  4.  Hybrid  automaton  model  in  SpaceEx  format  is  automatically  generated 
using  HyST  for  center-tapped  Buck  converter. 


[t>=  (1  -  D)  *T] 


Fig.  5.  SLSF  model  is  automatically  generated  using  HyST  for  center-tapped 
Buck  converter. 

The  corresponding  state  space  matrices,  during  the 
switching  cycle  DT  <  t  <  T,  are  thus  given  by 


'o 

_ n 

'o' 

II 

71 

L 

1 

,b2  = 

0 

c 

RC  _ 

We  skip  the  ODEs  for  the  third  mode  being  quite  straight¬ 
forward.  Using  HyST,  we  have  automatically  generated  the 
models  of  Buck  converter  based  on  above  ODEs  in  SpaceEx 
and  SLSF  formats  as  shown  in  Fig.  4  and  Fig.  5,  respectively. 
The  component  values  used  in  the  model  are  mentioned 
in  Fig.  3,  and  adopted  from  [12]. 

B.  Center-Tapped  Boost  Converter  Model 

It  is  a  special  type  of  DC-DC  boost  converter  with  a 
center-tapped  inductor  as  shown  in  Fig.  6.  As  in  the  above 
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Fig.  6.  Schematic  diagram  of  the  center-tapped  boost  converter. 


{il  =  0; 
vc  =  0; 
t  =  0; 
gt  =  0; 
mode  = 


'charging 

du: 

il_dot  =  b1c  *Vin; 
vc_dot  =  a22c  *  vc; 
t_dot  =  1 ; 
gt_dot=  1; 
mode_dot  =  0; 
il_out=il; 
vc_out=^/c; 
t_out=t; 
gt_out=gt; 
mode_out=mode; 

flyback_openloopJosses_locat'on  =1; 
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[t>=(1  -  D)  *T] 
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gt=gt 
vc  =  vc; 
il  =  il; 

mode  =  1 ;} 


[t>=D*T] 
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mode  =  2;} 


'discharging  ^ 

du: 

il_dot  =  a12o*vc; 

vc_dot  =  a21  o  *  il  +  a22o  *  vc; 

t_dot  =  1; 

gt_dot  =  1 ; 

mode_dot  =  0; 

il_out=il; 

vc_out=vc; 

t_out=t; 

gt_out=gt; 

mode_out=mode; 

■  flyback_openloop Josses  Jocation  =2; 

V _ _ _ 


Fig.  9.  SLSF  model  is  automatically  generated  using  HyST  for  flyback 
converter. 


Fig.  7.  Schematic  diagram  of  flyback  converter. 

case,  the  dynamics  of  the  circuit  depends  on  the  operation 
of  the  MOSFET  switch  resulting  in  two  modes.  We  have 
automatically  generated  the  models  of  center-tapped  boost 
converter  in  SpaceEx  and  SLSF  formats  using  HyST.  Due 
to  space  limitation,  we  skip  the  formulation  of  ODEs  and 
corresponding  model  figures.  The  component  values  used  in 
the  model  are  mentioned  in  Fig.  6. 

C.  Improved  Model  of  Flyback  Converter 

For  flyback  and  forward  transformer-isolated  DC-DC  power 
converters,  we  model  the  transformer  by  Lm,  a  parallel 
magnetizing  inductance,  at  the  input  side.  The  magnetizing 
current  through  Lm  is  denoted  by  i  [ym .  In  case  of  the  flyback 
converter  there  are  two  state  variables  (i.e.,  i[tTn  and  vc)  and 
two  modes.  A  simple  model  was  presented  in  [7]  for  this 
type  of  transformer-isolated  converter.  This  model  may  be 
improved  by  adding  an  ESR  (equivalent  series  resistor)  for 
the  capacitor  [13]  as  shown  in  Fig.  7.  For  space  limitation,  we 
skip  the  detailed  model  formulation.  We  have  automatically 
generated  SpaceEx  and  SLSF  models  of  flyback  converter 
as  shown  in  Fig.  8  and  Fig.  9,  respectively.  The  component 
values  used  in  the  model  are  mentioned  in  Fig.  7,  and  adopted 
from  [12], 


t  >=  (1  -  D)  *  T 


Fig.  10.  Schematic  diagram  of  forward  converter. 

D.  Improved  Model  of  Forward  Converter 

We  present  an  improved  model  of  the  forward  converter  that 
was  earlier  presented  in  [7]  to  include  the  MOSFET  switching 
loss  (modeled  by  a  series  resistance  rsw)  and  ESR  (rj  for 
the  inductor,  as  illustrated  in  Fig.  10.  There  are  three  state 
variables,  i.e.,  i [J ,  and  vc-  The  switching  modes  depend 

on  the  state  of  the  MOSFET  switch  as  well  as  the  fact  that 
whether  <  0  and  i <  0.  This  results  in  five  different 
modes  as  shown  in  Fig.  11  and  Fig.  12  for  SpaceEx  and  SLSF 
models,  respectively.  Due  to  space  limitation,  we  skip  the  ODE 
formulation.  The  component  values  used  in  the  model  are 
mentioned  in  Fig.  10. 

E.  Formal  Requirements  for  Verification  of  Power  Electronics 
Circuits 

Formal  verification  requires  that  a  given  model  of  a  power 
electronics  device  does  not  violate  a  predefined  stability 
specification.  We  use  the  Lyapunov  stability  to  define  this 


Fig.  8.  Hybrid  automaton  model  in  SpaceEx  format  is  automatically  generated 
using  HyST  for  flyback  converter. 


Fig.  11.  Hybrid  automaton  model  in  SpaceEx  format  is  automatically 
generated  using  HyST  for  forward  converter. 
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Fig.  12.  SLSF  model  is  automatically  generated  using  HyST  for  forward 
converter. 

specification,  i.e.,  x  =  f{x(t))  is  stable  ifVe>03<5>0 
such  that  if  ||x(0)||  <  5  =>  ||x(f)||  <  e  V  t  >  0.  Therefore, 
we  may  define  a  bounded  region  and  verify  that  the  output 
of  the  power  electronics  device  eventually  reaches  and  always 
remains  in  this  stable  region.  This  is  hypothetically  equivalent 
to  requiring  that  both  the  state  variables  of  interest,  i.e.,  i and 
vc  attain  a  stable  limit  cycle  in  finite  time.  Accordingly,  we 
define  the  stability  specification  for  DC-DC  power  converters 
in  steady  state,  such  that  i l  and  vc  should  attain  a  stable  limit 
cycle  within  a  finite  settling  time  ts- 

IV.  SLSF  Simulations  and  Reachability  Analysis 

We  have  automatically  generated  SpaceEx  models  using 
HyST  tool  and  analyze  these  in  SpaceEx  environment.  We 
have  also  automatically  translated  the  same  SpaceEx  models 
into  SLSF  format  using  HyST.  For  the  flyback  converter, 
we  require  that  vc  and  i^m  should  exhibit  a  stable  limit 
within  settling  time  ts-  For  the  center-tapped  Buck,  boost,  and 
forward  converters,  we  require  that  vc  and  is  should  exhibit 
a  stable  limit  within  settling  time  ts- 

For  center-tapped  Buck,  center-tapped  boost,  flyback,  and 
forward  converters,  the  SpaceEx  and  SLSF  results  for  the 
capacitor  voltage  and  inductor  current  are  shown  in  Fig.  13, 
Fig.  14,  Fig.  15,  and  Fig.  16,  respectively.  SLSF  simulation 
traces  are  contained  within  the  over-approximated  sets  of 
reachable  states  computed  using  SpaceEx.  We  also  conclude 
that  these  results  exhibit  stable  limit  cycle,  and  that  stable 
voltage  is  attained  within  1.5  ms,  5  ms,  3  ms,  and  2  ms  for 
the  respective  power  converters. 

V.  Conclusion 

HyST  significantly  reduces  the  time  and  efforts  in  model- 
based  design  process  and  formal  verification.  Verification  and 
validation  research  community  may  use  HyST  to  automatically 
transform  the  hybrid  automaton  models  in  SpaceEx  format  to 
other  formats  and  perform  reachability  analysis  using  aforesaid 
model  checking  tools.  The  hybrid  automaton  models  of  power 
electronics  circuits  that  we  provide  in  this  paper  form  part 
of  a  benchmark  library.  It  is  being  developed  to  evaluate 
various  reachability  analysis  and  verification  methods.  This 


benchmark  library  is  open  to  the  continuous  and  hybrid 
systems  verification  community  for  testing  and  evaluation  of 
their  methods  and  tools. 
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Fig.  13.  Comparison  of  SpaceEx  reach  sets  and  SLSF  trajectories  for  the  center-tapped  Buck  converter  showing  the  simulation  trace  containment  within 
overapproximated  sets  of  reachable  states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor 
current. 


Fig.  14.  Comparison  of  SpaceEx  reach  sets  and  SLSF  trajectories  for  the  center-tapped  boost  converter  showing  the  simulation  trace  containment  within 
overapproximated  sets  of  reachable  states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor 
current. 


< 


(a) 


Time,  Sec  ^q-3 


Fig.  15.  Comparison  of  SpaceEx  reach  sets  and  SLSF  trajectories  for  the  flyback  converter  showing  the  simulation  trace  containment  within  overapproximated 
sets  of  reachable  states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor  current. 


Fig.  16.  Comparison  of  SpaceEx  overapproximations  and  SLSF  trajectories  for  the  forward  converter,  showing  the  simulation  trace  containment  within 
overapproximated  sets  of  reachable  states:  (a)  Inductor  current  vs  time  (b)  Capacitor  voltage  vs  time  (c)  Phase-plane  plot  of  capacitor  voltage  and  inductor 
current. 
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Model  Validation  of  PWM  DC-DC  Converters 


Abstract — This  paper  presents  hybrid  automaton  mo¬ 
deling,  comparative  model  validation,  and  formal  verifica¬ 
tion  of  stability  through  reachability  analysis  of  PWM  DC- 
DC  converters.  Conformance  degree  provides  a  measure  of 
closeness  between  the  proposed  hybrid  automata  models 
and  experimental  data.  Non-determinism  due  to  variations 
in  circuit  parameters  is  modeled  using  interval  matrices. 
In  direct  contrast  to  the  unsound  and  computationally- 
intensive  Monte  Carlo  simulation,  reachability  analysis  is 
introduced  to  overapproximate  the  set  of  reachable  states 
and  ensure  stable  operation  of  PWM  DC-DC  converters. 
Using  a  200  W  experimental  prototype  of  a  buck  conver¬ 
ter,  hybrid  automata  models  of  open-loop  and  hysteresis- 
controlled  converters  are  first  validated  against  experimen¬ 
tal  data  using  their  conformance  degrees.  Next,  converter 
stability  is  formally  verified  through  reachability  analysis 
and  informally  validated  using  Monte  Carlo  simulations  and 
experimental  results. 

Index  Terms — DC-DC  converter,  formal  verification,  hy¬ 
brid  automaton,  model  validation,  reachability  analysis. 

I.  Introduction 

BSTRACT  models  of  PWM  DC-DC  converters  should 
reasonably  match  the  experimental  data  obtained  from 
a  hardware  prototype  despite  parametric  uncertainty.  More- 
efficient  stochastic  simulation  techniques  are  based  on  po¬ 
lynomial  chaos,  where  parametric  uncertainties  are  accoun¬ 
ted  for  by  a  series  of  orthogonal  polynomials  that  depends 
upon  their  probability  distributions  [1],  Series  coefficients  are 
computed  using  various  intrusive  (e.g.,  stochastic  Galerkin 
[1])  or  non-intrusive  (e.g.,  stochastic  collocation  [2])  methods. 
Examples  of  such  stochastic  methods  for  electrical  circuits 
and  power  systems  include  Galerkin-based  generalized  poly¬ 
nomial  chaos  [3],  SPICE-compatible  stochastic  Galerkin  [4], 
Galerkin-based  generalized  decoupled  polynomial  chaos  [5], 
stochastic  testing  [6],  and  SPICE-compatible  stochastic  collo¬ 
cation  approach  [7].  In  general,  polynomial  chaos  methods 
suffer  from  the  curse  of  dimensionality,  slow  convergence 
with  discontinuous  solutions,  and  substantial  computational 
overhead  [8] — [1 1]. 

Another  conventional  approach  is  the  simulation-based 
Monte  Carlo  paradigm  [12],  [13],  wherein  considering 
all  possible  parameter  variations  and  initial  conditions  is 
computationally-prohibitive.  Moreover,  for  a  higher  level  of 
confidence  in  results  produced  by  the  Monte  Carlo  analysis, 
greater  number  of  simulation  runs  are  required.  Generally, 
the  total  number  of  Monte  Carlo  simulations,  cr,  has  to  be 
increased  by  100-fold  to  achieve  additional  decimal  place 
of  precision,  owing  to  the  O(^)  convergence  rate  [14], 
Conceptually,  to  have  a  full  confidence  in  Monte  Carlo  results, 
one  would  require  infinite  number  of  simulation  runs  [15], 
[16],  The  level  of  required  modeling  fidelity  depends  on  the 
critical  nature  of  the  application  domain.  For  example,  the 
root  cause  of  the  2014  recall  of  around  700,000  Toyota  Prius 


rs  r 


Fig.  1 .  Closed-loop  DC-DC  buck  converter  with  main  parasitic  elements. 

cars  was  attributed  to  an  error  in  the  interaction  between  a 
boost  converter  and  its  software  controller  [17].  Likewise, 
more  than  100,000  Toyota  Prius  cars  were  recalled  due  to 
an  inverter  failure  [18].  Therefore,  this  mission-critical  domain 
would  require  significant  confidence  in  the  modeling  accuracy. 
At  the  same  time,  the  utilized  model  validation  tool  should  be 
conservative  enough  to  overapproximate  all  possible  sets  of 
states  reachable  by  the  model  execution. 

The  formal  verification  community  has  been  using  reacha¬ 
bility  analysis-based  model  checking  tools  to  have  sufficient 
confidence  in  the  model.  Therefore,  we  first  use  rigorous 
model  validation  paradigms  [19]  to  quantify  the  closeness 
between  the  abstract  model  waveforms  and  experimental  data 
using  the  conformance  degree  [20].  Stable  converter  operation 
is  then  formally  verified  on  the  model  using  reachability 
analysis.  The  boundaries  of  state  trajectories  can  be  found 
from  average-value  models  [21],  [22],  Reachability  analysis 
overapproximates  the  set  of  all  possible  reachable  states  (i.e., 
the  reach  sets)  from  a  given  set  of  initial  states  and  parameter 
values.  One  can  then  confidently  ascertain  a  stable  converter 
operation  if  the  reach  sets  remain  within  a  desired  region 
of  the  state  space  for  a  given  time  span.  Without  loss  of 
generality,  we  have  considered  a  DC-DC  buck  converter,  with 
main  parasitic  elements,  as  shown  in  Fig.  1. 

General  reachability  analysis  tools  include,  but  are  not 
limited  to,  HyTech  [23],  PHAVer  [24],  UPPAAL  [25],  HSolver 
[26],  d/dt  [27],  Flow*  [28],  and  SpaceEx  [29].  To  effectively 
use  such  model  checking  tools,  hybrid  automata  models  of 
DC-DC  converters  are  required  [30].  Hybrid  automaton  mo¬ 
deling  of  DC-DC  converters  is  presented  in  [3 1  ]— [36] .  Ho¬ 
wever,  [33]— [35]  do  not  consider  component  losses/variations 
and  the  discontinuous  conduction  mode  (DCM),  and  do  not 
perform  the  reachability  analysis.  PHAVer  in  [37]  computes 
the  reach  sets  for  an  open-loop  boost  converter  but  does 
not  include  DCM  or  component  losses.  MATLAB/Ellipsoidal 
Toolbox  is  used  in  [38]  for  the  reachability  analysis  of  DC-DC 
converters.  However,  Ellipsoidal-based  set  computations  suffer 
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from  the  curse  of  dimensionality.  SpaceEx  (the  successor  of 
PHAVer)  scales  quite  efficiently  and  is  used  as  the  reachability 
analysis  tool  in  this  paper. 

We  have  formally  defined  a  precise  hybrid  automaton  model 
for  PWM  DC-DC  converters,  that  accommodates  main  circuit 
parasitics,  DCM,  and  the  non-determinism  due  to  parameter 
variations,  that  have  not  been  considered  altogether  in  any 
past  work.  We  also  use  the  notion  of  conformance  degree 
to  compare  different  model  abstractions,  using  their  output 
trajectories,  that  has  not  been  used  in  any  of  the  work 
cited  above.  Moreover,  all  the  hybrid  automata  models  are 
automatically  generated.  Herein,  the  proposed  approach  is 
shown  to  outperform  the  traditional  Monte  Carlo  simulation 
in  computation  time.  In  summary,  the  main  contributions  of 
this  paper  are: 

•  Hybrid  automata  models  for  DC-DC  converters  are  auto¬ 
matically  generated,  validated  against  Simulink/Stateflow, 
PLECS  simulations,  and  hardware  measurements,  and 
verified  using  reachability  analysis  in  SpaceEx.  These 
models  include  component  nonidealities  and  different 
operational  modes. 

•  The  conformance  degree  of  the  hybrid  automata  models 
validates  these  against  the  experimental  data,  by  provi¬ 
ding  a  proximity  measure  between  executions/behaviors 
of  these  two  in  both  time  and  space. 

•  Non-determinism  due  to  parametric  variations  is  modeled 
using  interval  matrices,  which  results  in  a  set-valued 
additive  input  term  in  the  system  dynamics. 

•  The  reachability  analysis  achieves  a  fixed  point  where 
there  are  no  other  reach  sets  (i.e.,  the  model  output  will 
remain  within  reach  sets  as  t  — >  oo).  It  is  impossible  to 
get  such  success  through  Monte  Carlo  analysis. 

The  remainder  of  this  paper  is  organized  as  follows:  Hybrid 
automaton  modeling  is  discussed  in  Section  II.  Application 
of  conformance  degree  for  model  validation  is  discussed 
in  Section  III.  Section  IV  uses  interval  analysis  to  mo¬ 
del  the  non-determinism  caused  by  the  parameter  variation. 
SpaceEx-based  reachability  analysis  is  discussed  in  Section  V. 
Section  VI  validates  the  developed  models  against  a  200  W 
buck  converter  prototype  using  the  conformance  degree,  for¬ 
mally  verifies  the  model  properties  using  reachability  analysis, 
and  presents  comparison  with  the  Monte  Carlo  simulation. 
Section  VII  concludes  the  paper. 

II.  Hybrid  Automaton  Modeling 

A.  Preliminaries 

DC-DC  converters  exhibit  both  continuous  and  discrete  be¬ 
haviors  due  to  the  presence  of  passive  elements  and  switching 
components,  respectively.  Hybrid  automaton  modeling  [39] 
integrates  resulting  differential  equations  and  finite  state  ma¬ 
chines  in  a  single  formalism.  The  state  of  a  hybrid  automaton 
model  may  change  in  two  ways,  i.e.,  through  a  continuous 
flow  trajectory  within  a  given  topology  (Definition  2.2),  and 
through  a  discrete  transition  between  two  given  topologies 
(Definition  2.3).  A  topology  is  defined  as  the  circuit  confi¬ 
guration  in  each  switching  sub-interval  (Fig.  2).  We  define 
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Fig.  2.  Topologies,  operational  modes,  and  hybrid  automaton  modeling 
of  a  DC-DC  buck  converter. 

R”  as  the  set  of  n-dimensional  reals,  and  2X  as  the  power  set 
of  a  given  set  A',  i.e.,  the  set  of  all  the  subsets  of  A. 

B.  Hybrid  Automaton  Model  Syntax  and  Semantics 

We  first  formally  define  the  model  components  in  mathema¬ 
tical  set  representation,  and  then  define  the  model  execution 
as  these  components  interact. 

Definition  2.1:  A  hybrid  automaton  model  is  defined  by  a 
tuple  TL  =  ( Q,X,init,U,E,g,G,inv,h,F ),  which  has  the 
following  components: 

•  Topologies:  Q  =  {<7i,  92>  ••••)  Qn}  is  a  finite  set  of 
topologies. 

•  State  Variables:  X  C  R"  is  set  of  continuous  state 
variables.  A  state  is  defined  by  ( q ,  x)  £  Q  x  X. 

•  Initial  Conditions:  init  C  Q0  x  Ao  is  a  set  of  initial 
conditions,  such  that  Qo  C  Q  and  Ao  C  A. 

•  Inputs:  U  =  ...un}  is  the  set  of  inputs  for  each 

topology. 

•  Discrete  Transitions:  E  C  Q  x  Q  is  a  set  of  feasible 
discrete  transitions  allowed  among  the  topologies,  such 
that  an  element  eij  =  ( qi ,  qfi)  £  E  implies  that  a  discrete 
transition  from  ith  topology  to  jth  topology  is  allowed.  It 
might  not  be  possible  to  visit  the  entire  set  of  topologies 
from  one  particular  topology  (Definition  2.3). 

•  Guard  Function:  g  :  E  — ►  G  is  the  guard  function  that 
maps  each  element  eL3  £  E  to  its  corresponding  guard 


g{eij )  £  G. 

Guards:  G  C  2X  is  the  guard  set  such  that  3  g(ei3)  £  G 
for  each  etj  £  E.  A  guard  is  a  property  of  the  hybrid 
automaton  model  that  must  be  satisfied  by  a  state  to  take 
a  discrete  transition  from  a  given  topology  to  another 
pre-defined  topology.  A  state  ( qk,xk )  £  Q  x  A  satisfies 
g(eij)  (i.e,  (qk,xk)  1=  g{ei:j))  iff  qk  =  q.t  and  xk  £  g(ei:j). 
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Fig.  3.  Execution  of  the  hybrid  automaton  model  of  DC-DC  converters. 

•  Invariants:  inv  :  Q  — ►  2X  is  a  mapping  that  assigns 
an  invariant  inv  ( q )  C  A'  for  each  topology  q  £  Q.  An 
invariant  is  a  property  of  the  hybrid  automaton  model  that 
must  be  satisfied  by  all  the  states  for  a  given  topology. 
A  state  (q,x)  \=  inv(q)  iff  x  £  inv(q). 

•  Reset  of  Continuous  State:  h  :  E  x  X  — >  X  resets  the 
continuous  state,  i.e.,  if  a  discrete  transition  takes  place 
from  ith  topology  to  jth  topology  as  defined  by  £  E 
with  x  £  X,  the  continuous  state  is  reset  to  a  new  value 
x'  =  h(eij,x)  £  A,  such  that  x'  £  inv(qj). 

•  Set  of  ODEs:  F  is  the  set  of  ordinary  differential  equa¬ 
tions  (ODEs)  that  are  defined  for  each  topology  q  £  Q 
over  the  continuous  variables  x  £  X.  The  continuous 
dynamics  for  each  q  £  Q  is  defined  by  F(q,x,u)  over 
a  given  time  horizon  t  £  [r1;  T2]  that  assigns  a  Lipschitz 
continuous  vector  space  in  R". 

Remarks:  Here,  x'  £  X  symbolizes  the  new  value  of  a 
continuous  state  x  £  X  after  a  continuous  flow  or  a  discrete 
transition.  If  a  state  (q,  x)  does  not  satisfy  an  invariant  inv(q), 
then  real  time  r  is  stopped,  forcing  the  continuous  state 
x  to  stop  evolving  within  a  topology.  The  guard  function 
ensures  discrete  transition  to  an  appropriate  topology,  once  the 
corresponding  guard  is  satisfied.  Here,  invariants  and  guards 
are  defined  in  the  form  of  bounds  over  continuous  state 
variables  in  Fig.  3. 

Definition  2.2:  The  continuous  flow  trajectory  T  for  a 
hybrid  automaton  model  j-L  is  defined  by  the  valuations  of 
x  £  X.  For  a  given  initial  state  (q,  xq)  £  Q  x  X  and  u  £  U, 
3  f(q ,  x,u)  £  F  that  results  in  a  final  continuous  state  x'  £  X, 
whereas  q  remains  unchanged  with  given  invariant  inv(q ),  iff 
(q,x)  N  inv(q).  V  t  £  [tt  t2],  T  is  given  by 

T2 

T(q,x')  =  x0  +  J  f(q,x,u)dt.  (1) 

Tl 

f 

and  denoted  by  (q,x 0)  - >  ( q,x '). 

At  each  topology,  converter  dynamics  can  be  modeled 
by  ODEs;  e.g.,  system  matrices  Aq  and  Bq  describe  the 
continuous  flow  trajectories  in  topology  q  £  {1, 2, 3}  of  Fig.  2. 

Definition  2.3:  The  discrete  transition  for  a  hybrid  automa¬ 
ton  model  TL  is  defined  as:  for  a  given  state  ( r/, ,  x)  £  Q  x  X 


and  u  £  U,  there  is  a  function  hfeij,x )  that  resets  the 
continuous  state  to  x'  £  X,  and  the  topology  to  qj,  iff 
( qi,x )  t=  inv{qf)  and  (qi,x)  t=  g(eij)  £  G,  and  3  etj  £  E. 
The  discrete  transition  is  denoted  by  (qt,x)  — ^A  ( qj,x' ). 

Definition  2.4:  An  execution  of  a  hybrid  automaton  model 
'H  is  an  alternating  sequence  of  continuous  flow  trajectories 
and  discrete  transitions. 

The  example  of  an  execution  is  shown  in  Fig.  3. 

The  switching  instance  can  be  determined  either  externally 
(e.g.,  by  a  duty  cycle  command  for  the  MOSFET)  or  internally 
(e.g.,  by  meeting  appropriate  threshold  conditions  for  the 
diode).  The  sequence  of  topologies,  observed  periodically  in 
the  steady  state,  defines  an  operational  mode.  Example  of  three 
topologies  and  two  operational  modes  for  a  buck  converter  are 
shown  in  Fig.  2. 

C.  Model  Instantiation  for  DC-DC  Converters 

We  may  now  implement  the  syntax  and  semantics  of  the 
hybrid  automaton  model  developed  above  for  DC-DC  conver¬ 
ters.  We  define  D  as  the  duty  cycle,  Tsw  as  the  switching 
period,  and  V)n  as  the  DC  input  voltage.  We  can  represent  the 
continuous  dynamics  for  a  given  topology  as  a  standard  set  of 
state-space  equations 

^  =  AqX  +  BqU  (2) 

where,  x  £  R"  is  a  vector  of  continuous  states,  Q  is  a 
finite  set  of  topologies,  u  C  U  such  that  U  C  Rm  is  a 
set  of  input  vectors,  and  Aq  £  Rraxn  and  Bq  £  Rnxm 
are  system  matrices.  Such  formation  can  be  readily  created 
for  the  buck  converter  in  Fig.  2.  The  instantiation  of  the 
hybrid  automation  model  for  an  open-loop  DC-DC  converter, 
as  per  Definition  2.1,  is: 

•  Three  topologies  are  denoted  by  Q  =  {91, <72, 93}- 

•  The  continuous  state  vector  is  x  =  [i^  vq  t]7,  where  r 
represents  real  time  such  that  =  1. 

•  U  =  {[Vin,  0,  0]7,  [0,  0,  0]7,  [0, 0, 0]7}  forms  the  input 
vector  set. 

•  E  =  {(<?i,  92),  (<72,  gi),  (<72,  93),  (93,91)}  defines  the 
feasible  discrete  transitions,  e.g.,  (q2l  qf)  means  a  discrete 
transition  from  topology  2  to  3  is  allowed. 

•  Guard  set,  for  the  corresponding  elements  of  E,  is  defined 

by  G  =  {(r  >  DTSW) ,  (r  >  (1  —  D)  Tsw) ,  [iL  <  0) , 
(r>  (1  —  D)TSW)}. 

•  The  continuous  flow  trajectory  is  defined  by  (2),  with 
the  corresponding  state  matrices  for  each  topology.  For 

f 

topology  1,  this  can  be  denoted  by  (gi,a:o)  -1 (qi,x'), 
as  shown  in  Fig.  3.  Here,  (qi,Xo)  is  the  initial  state  and 
(qi ,  x')  is  the  final  state  as  the  automaton  continuously 
evolves  with  the  continuous  flow  dynamics  fi(x). 

•  The  reset  function  h  defines  a  new  continuous  state  x"  for 
the  new  topology.  For  example,  if  a  transition  is  to  take 
place  from  topology  1  to  topology  2  with  some  final  state 
x '  £  X'  C  X  in  topology  1,  h  assigns  the  new  state  x"  £ 
X"  C  X  in  topology  2.  For  topology  1  to  topology  2,  a 
discrete  transition  is  denoted  by  (qi,xr)  — (q2,x"),  as 
shown  in  Fig.  3. 
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Fig.  4.  Output  trajectories  of  capacitor  voltage  for  the  closed-loop 
controlled  buck  converter  -  local  mismatch  for  interval  rc  and  e. 

The  evolution  of  the  hybrid  automaton  model  starts  with 
initial  conditions  from  set  init,  e.g.,  (<71,  Xo)  G  init  for  a 
given  input  ui  =  [Vin,  0,0]'  and,  subsequently,  the  continuous 
state  evolves  according  to  the  set  of  ODEs  defined  by  (2) 
(i.e.,  F  in  Definition  2.1).  The  topology  remains  the  same,  i.e., 
q  ( t )  =  qi,  as  xq  evolves  inside  the  invariant  inv  (qr),  such 
that  it  attains  a  final  value  x'  £  inv  (qi).  Once  the  continuous 
state  x'  satisfies  the  guard  g  (egiq2)  corresponding  to  the  edge 
eqiq2  £  -E,  the  topology  may  transition  from  q\  to  <72,  and 
the  continuous  state  is  reset  with  a  new  value  x"  in  the  new 
invariant  set  inv  ((72)  C  X  with  a  new  input  U2  =  [0,  0,  0]'. 

This  hybrid  automaton  model  can  be  extended  to  closed- 
loop  DC-DC  converters,  e.g.,  hysteresis-controlled  converters. 
The  tuple  remains  the  same  except  that  the  guards  are  defined 
in  terms  of  switching  boundaries.  The  hysteresis  band  is 
formed  by  defining  an  upper  switching  boundary,  Vref  +  8 , 
and  a  lower  switching  boundary,  Vref  —  6,  where  Vref  is 
the  desired  output  voltage,  and  8  is  the  tolerance  level. 
Thus,  G  =  {(uC  >  Vref  +  5)  ,  ( VC  <  Vref  ~  8)  ,  (lL  <  0)  , 
{VC  <  Vref  ~  #)}. 

It  should  be  noted  that  time  r  does  not  appear  in  the 
guard  expressions.  Therefore,  we  have  developed  two  hybrid 
automata  models  for  the  closed-loop  buck  DC-DC  converter, 
i.e.,  one  with  variable  r  (called  the  time-dependent  hybrid 
automaton  model),  and  another  without  variable  r  (called 
the  time-independent  hybrid  automaton  model).  For  the  time- 
independent  hybrid  automaton  model,  we  perform  the  reacha¬ 
bility  analysis  for  an  unbounded  time,  i.e.,  compute  the  reach 
sets  as  t  — >  oo. 

III.  Validation  through  Conformance  Degree 

Model  validation  of  DC-DC  converters  requires  comparing 
output  trajectory  as  defined  by  (1)  for  a  given  hybrid  auto¬ 
maton  model  XL  with  the  measured  data  from  an  experimental 
prototype  referred  to  as  X. 

Our  goal  is  to  find  an  appropriate  measure  of  distance  for 
output  trajectories  of  hybrid  automata  models.  One  can  consi¬ 
der  the  output  trajectories  of  the  capacitor  voltage  ( vc )  for  a 
closed-loop  buck  converter  shown  in  Fig.  4.  The  experimental 
data  obtained  from  a  prototype  and  output  trajectory  of  the 
hybrid  automaton  model  in  Simulink/Stateflow  are  overlaid. 


Intuitively,  the  two  output  trajectories  look  similar;  however, 
the  sup  norm  would  give  a  large  value  to  the  distance  between 
them.  This  is,  partly,  because  X  and  XL  might  transition 
among  various  topologies  at  slightly  different  moments  in 
time.  Therefore,  our  distance  measure  should  allow  some 
wiggle  room  in  time.  Rather  than  comparing  only  the  states 
that  are  exactly  time-aligned,  it  should  allow  comparison  of 
states  that  are  within  some  rc  >  0  time  units  of  each  other. 

Moreover,  it  is  not  appropriate  to  compare  outputs  when 
two  systems  have  executed  different  numbers  of  discrete 
transitions.  Thus,  our  distance  measure  must  only  compare 
states  after  an  equal  number  of  discrete  transitions  between 
topologies  of  the  two  systems.  Note  that  within  the  time 
window  tc  in  Fig.  4,  both  the  hardware  prototype  as  well 
as  the  Stateflow  model  exhibit  two  discrete  transitions.  To 
this  end,  we  introduce  the  parameter  j  £  N,  that  counts  the 
number  of  discrete  transitions  each  system  makes,  where  N  is 
the  set  of  natural  numbers.  It  is  reasonable  to  require  that  the 
transition  times  of  the  two  systems  be  close  to  consider  that 
the  systems  themselves  are  close:  the  value  tc  will  also  bound 
the  difference  in  transition  times.  The  distance  measure  will 
account  for  the  distance  between  output  trajectories,  captured 
by  the  value  e  >  0.  Thus,  we  have  a  2-value  distance  measure, 
with  values  rc  and  e  capturing  the  time  and  space  distance 
between  the  two  output  trajectories  as  illustrated  in  Fig.  4. 

The  output  trajectories  of  hybrid  automata  models  are 
parameterized  with  t  and  j.  The  time  spent  in  a  given  converter 
topology  is  t  £  R>o,  and  j  £  N  counts  the  number  of  discrete 
transitions  between  different  topologies  (where  LT:>0  is  the 
set  of  positive  real  numbers).  We  write  y(t,j)  for  the  output 
trajectory  at  the  hybrid  time  (t,j)  £  R>0  x  N,  i.e.,  at  time 
t  and  after  j  transitions.  Fet  domy  C  R>o  x  N  denote  the 
domain  of  output  trajectory  y,  i.e.,  the  set  of  all  (t.  j),  so  that 
(T,  J,  rc,  e)-closeness  [20]  can  be  formally  defined. 

Definition  3.1:  Take  an  output  trajectory  for  time  T  £  R>o, 
a  maximum  number  of  discrete  transitions  J  £  N,  and 
parameters  rc,e  >  0.  Two  output  trajectories  y\  and  y 2  are 
(T,  J,  tc,  £)-close,  shown  as  y\  ~(Tc,e)  yi ,  if  (a)  for  all  (t.  j)  £ 
domj/i  such  that  t  <  T,  j  <  J,  there  exists  (s,j)  £  domy2 
where  \t  -  s\  <  rc,  and  \\yi(t,j)  -  y2(s,  j)||  <  e,  and  (b)  for 
all  (s,  j)  £  dom?/2  such  that  s  <  T,  j  <  J,  there  exists  (t,j)  £ 
domy!  where  \t  -  s\  <  rc,  and  || y2(s,j)  -  Vi(t,j)\\  <  £. 

(T,  J,  rc,  e)-closeness  gives  a  proximity  measure  between  the 
two  output  trajectories  in  both  time  and  space.  It  shows  that  for 
every  point  yi(t,j),  y2  has  a  point  y2{s,j)  which  is  e-close  to 
it,  and  may  occur  anywhere  in  the  window  [t  —  Tc,t  +  Tc]  (and 
vice  versa).  Allowing  this  wiggle  room  in  time  is  important 
when  comparing  the  output  trajectories,  because  the  discrete 
transitions  could  occur  at  different  times.  The  two  values  T 
and  J  limit  our  testing  horizon.  (T,  J,  rc,  e)-closeness  can  be 
lifted  from  output  trajectories  to  systems.  One  can  validate 
the  model  through  the  conformance  degree  between  its  output 
trajectory  and  measured  data. 

Definition  3.2:  Fet  XL\  and  Xi2  be  two  hybrid  automata 
models.  The  conformance  degree  of  XL\  to  XL2.  given  rc,  is 
defined  as  the  smallest  e  such  that  for  every  trajectory  y\  of 
XLi,  there  exists  a  trajectory  y2  of  XL2 ,  where  y-\  ~(Tc,e)  2/2- 
We  denote  this  conformance  degree  by  CDt{XLi,XL2). 
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We  will  use  this  definition  intuitively  for  model  validation 
of  DC-DC  converters.  We  compute  the  conformance  degree 
CDT(fti,ft2)  f°r  some  rc  >  0  in  different  case  studies  of 
Section  VI,  and  effectively  say  that  some  local  mismatch  is 
permissible  within  a  window  rc  for  the  output  trajectories  of 
the  models  and  the  hardware  prototype. 


IV.  Modeling  Non-Determinism  using  Interval 
Analysis 

The  system  matrices  in  the  hybrid  automata  models  of  DC- 
DC  converters  depend  on  component  values.  The  variations 
due  to  manufacturing  tolerance,  aging,  and  temperature  result 
in  non-determinism  of  component  values.  Analysis  of  electri¬ 
cal  circuits  with  such  variations  has  been  reported  in  literature 
using  interval  arithmetic-based  genetic  optimization  [40]  and 
affine  arithmetic  [41],  We  use  the  interval  arithmetic  [42]  to 
incorporate  the  parameter  variations  within  the  reachability 
analysis  framework.  The  range  of  component  values  are  re¬ 
presented  in  terms  of  intervals.  A  real  interval  v  is  a  set  of 
real  numbers  given  by 


[u,  u]  =  {u  €  IR  :  v  <  v  <  u},  (3) 


where  v  is  the  infimum  and  v  is  the  supremum.  Given  two 
intervals,  [u.  w]  and  [u,  v],  their  product  is  another  interval 
given  by 


[u,  u]  *  [u,  v\  =  [ min(uv ,  uv,  uv,  uv) ,  max(uv ,  uv,  uv,  uv)] . 

(4) 

The  quotient  of  two  intervals,  with  a  non-zero  divisor,  is 


[u,u\ 

[u,T] 


=  [u,  u]  * 


where 
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If  [t>,  v\  has  both  bounds  negative,  then 


(5) 

(6) 


n  state  variables  having  single  deterministic  input  Vin,  with 
the  following  state-space  representation 
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Clin 

a2n 

Xi 

X2 

V 

^2 
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(10) 

We  use  SpaceEx  reachability  analysis  tool  (discussed  in 
Section  V)  to  compute  the  reach  sets  for  non-deterministic 
hybrid  automaton  model,  with  linear  dynamics  defined  by  (10) 
for  a  given  topology.  It  may  be  mentioned  that  SpaceEx,  in 
its  present  version,  does  not  fully  handle  the  matrix  algebra 
operations.  Hence,  we  need  to  define  the  state  dynamics  as 
scalar  combination  of  other  state  variables.  For  example,  for 
the  ith  state  variable  in  (10),  one  has 

X%  —  CLi 1 X  i  T  a?;2  X2  T  •  •  •  T  U ij Xj  T  . . .  4“  Clin Xn  T  bi  Vin  •  (11) 

To  incorporate  parameter  variation,  one  can  replace  the  above 
coefficients  with  intervals,  and  write  the  expression  as  a 
differential  inclusion 

X i  G  [tt-il ,  T  . ..  [Uij  ,  Qij^Xi . . .  A  [a  in  ■  &in]Xn  T  (l]  Vjn . 

(12) 

Since  SpaceEx,  in  its  present  version,  does  not  support 
the  interval  arithmetic,  the  intervals  [a.^ ,  ojj]  of  (12)  are 
computed  outside  the  SpaceEx  environment  using  (4),  (5), 
(6),  and  (7).  Subsequently,  these  intervals  are  transformed 
into  the  midpoint-radius  representation  to  include  the  state 
and  parametric  intervals  before  implementing  in  the  SpaceEx 
environment.  Using  (8)  and  (9),  one  can  write  (12)  in  a 
midpoint-radius  representation  as 

ii  €  {mid(an)  ±  rad(aii)}xi  +  ...  +  {mid(aij)  ± 
rad(ciij)}xj  +  ...  +  {mid(ain)  ±  rad{a.in)} xn 
+  {mid(bi)  ±  rad(bi)}Vin.  (13) 

The  mid-points  correspond  to  the  nominal  parameter  values 
that  are  constant  terms,  which  can  be  separated  as 

ii  G  (anxi  +  ra)  +  ...  +  ( ciijXj  +  rv,-)  +  ...  + 

( UinXn  T  Tin  )  +  ( biVin  +  Tbi )•  (14) 


(  1  V 

'1 

1 

V  [v,  v]  J 

—  5 

y 

V 

These  intervals  may  also  be  defined  by  the  midpoint-radius 
representation 

mid[v)  =  —(v  +  v),  (8) 

rad{v)  =  —(v  —  v).  (9) 

The  interval  matrix  for  the  system  matrix  is  A  =  [A.  ,4]. 
System  stability  can  be  deferred  by  examining  matrix  extrema, 
i.e.,  A  and  A  [43].  Therefore,  it  is  sufficient  to  consider 
every  combination  of  matrix  extrema  to  overapproximate  the 
reach  set.  The  overapproximation  of  an  interval  matrix  A  is 
given  by  splitting  it  into  two  parts,  i.e.,  a  nominal  part  and  a 
symmetric  part  [44].  Consider  a  linear  dynamic  system  with 


This  defines  the  continuous  dynamics  in  the  hybrid 
automaton  model  for  the  state  variable  xt.  The  radii 
Tn,Ti2, ...,  Tij ,  ...,Tin,  and  Tbi  are  expressed  as  product  of  the 
state  and  parametric  intervals,  such  that  rtj  is  given  by 

Tij  G  [—rad(aij),rad(a,ij)}  *  [xj,xj],  (15) 

where,  Xj  varies  between  Xj  and  TJ.  For  example,  for  the 
hysteresis-controlled  DC-DC  buck  converter  considered  here, 
vc  =  0  V  and  vc  =  20  V  in  (15).  Thus  the  coupling 
between  the  state  variables  is  accommodated  in  the  amended 
SpaceEx  model  by  formulating  in  terms  of  [xj,xj],  and 
incorporating  it  in  the  dynamics  in  (14).  The  product  of  the 
two  intervals  in  (15)  is  yet  another  interval,  obtained  using 
(4).  The  intervals  thus  computed  are  used  in  the  model  to 
define  the  lower  and  upper  bounds  for  respective  radii.  Since 
this  treatment  of  the  state  variables  as  intervals  is  not  catered 
in  Monte  Carlo  simulations,  SpaceEx  provides  more  reliable 
results. 
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Fig.  5.  Reachability  analysis  using  reach  sets  for  formal  verification  of  a 
hybrid  automaton  model. 

V.  Reachability  Analysis  for  Hybrid  Automata 

Reachability  analysis  has  been  used  by  the  formal  verifica¬ 
tion  community,  and  we  have  implemented  it  in  its  entirety 
for  PWM  DC-DC  converters  modeled  as  hybrid  automata.  In 
general,  reachability  analysis  has  been  documented  to  produce 
more  reliable  results  than  Monte  Carlo  simulations: 

1.  The  reachability  analysis  is  more  efficient.  Monte  Carlo 
analysis  becomes  computationally  less  tractable  with  in¬ 
creased  size  and  complexity  of  a  given  system  [38], 

2.  Reachability  analysis  is  conclusive.  In  contrast,  infinitely 
many  Monte  Carlo  simulations  are  required  to  span  the 
entire  design  parameter  space  and  operational  conditions 
and  have  full  confidence  in  the  final  results  [15],  [16]. 

3.  SpaceEx-based  reachability  analysis  considers  the  entire 
state  space  [45],  while  Monte  Carlo  simulations  only 
sample  the  parameter  space.  Generally,  reachability  ana¬ 
lysis  is  theoretically  superior  and  more  sound  [46]. 

We  formally  verify  the  stability  properties  of  non- 
deterministic  hybrid  automata  models  of  PWM  DC-DC  con¬ 
verters  through  the  reachability  analysis.  We  define  the  sta¬ 
bility  in  the  sense  of  Lyapunov,  i.e.,  x  =  f(x(t))  is  stable 
ifV0>O,3/3>O  such  that  if  ||x(0)||  <  (3  =>  ||x(f)||  < 
9  V  t  >  0.  We  may  define  a  bounded  region  and  verify  that  the 
output  of  the  hybrid  automaton  model  eventually  reaches,  and 
always  remains,  in  this  stable  region,  as  seen  in  Fig.  5.  We 
define  the  stability  specification  such  that  from  the  settling 
time  ts,  the  output  voltage  Vc{t)  should  remain  bounded 
within  a  tolerance  7  of  the  reference  voltage  Vref(t),  i.e.,  for 
t>ts=>  Vc{t)  =  Vref(t)  ±  7. 

Definition  5.1:  State  x  is  reachable  iff  3  an  execution  a 
such  that  x  £  a. 

Definition  5.2:  The  set  of  reachable  states  contains  all  the 
states  that  are  reachable  from  a  given  set  of  initial  conditions 
for  a  given  time. 

Consider  an  example  of  an  autonomous  system  x  =  Ax. 
The  set  of  reachable  states  from  initial  time  to  to  final  time 
tf,  from  a  given  initial  set  Xq,  is 


Fig.  6.  Reach  sets  in  different  topologies  with  transitions  imposed  by 
guards. 

However,  (16)  does  not  cater  to  the  discrete  transitions  asso¬ 
ciated  with  the  hybrid  dynamical  systems.  Additionally,  the 
exact  set  of  all  reachable  states  is  undecidable  [29]. 

In  practice,  overapproximations  of  the  reachable  states  are 
computed  using  geometrical  data  structures  (e.g,  boxes,  po¬ 
lytopes,  ellipsoids,  or  zonotopes  [47]),  and  denoted  by  TZ. 
For  simplicity,  we  call  these  overapproximations  as  the  reach 
sets  in  this  paper.  This  framework  can  be  extended  to  hybrid 
dynamical  systems  by  including  invariants  and  guard  sets  (Fig. 
6),  and  implemented  in  various  reachability  analysis  tools  by 
software  research  community  as  mentioned  in  Section  I.  The 
reach  sets  for  continuous  dynamics  can  be  computed  using 
continuous  post-operators  so  long  as  the  continuous  dynamics 
of  DC-DC  converter  are  contained  within  the  invariant  set 
defined  for  the  corresponding  topology  or  do  not  enter  the 
guard  set.  Once  the  guard  condition  is  satisfied  within  an 
invariant,  a  transition  takes  place  from  topology  1  to  topology 
2  such  that  the  next  reach  set  is  computed  using  discrete  post¬ 
operator.  This  process  goes  on  until  either  the  final  time  in  a 
local  time  horizon,  or  a  fixed  point,  is  reached.  A  fixed  point 
signifies  that  the  reachability  algorithm  cannot  find  any  new 
reach  set  during  the  current  iteration  other  than  those  computed 
in  the  previous  iteration.  SpaceEx  reachability  tool  computes 
the  reach  sets  of  a  hybrid  dynamical  system.  It  is  a  classical 
fixed  point  algorithm  based  on  computation  of  symbolic  states 
[29], 

Definition  5.3:  A  symbolic  state  is  defined  as  a  pair  (<7,  0), 
where  q  is  a  topological  instance,  and  0  is  the  corresponding 
convex  continuous  set. 

The  reach  set  1Z  is  obtained  by  computing  the  set  of 
symbolic  states.  This  reach  set  is  the  fixed  point  of  the 
sequence  7 Z0  =  postc  ( I  nit ),  and  the  successors  are 

IZk+i  :=  lZk  [J  postc  ( postd  (7 Zk))  (17) 

where,  postd  is  the  discrete  post-operator  that  defines  the 
reach  sets  after  a  discrete  transition  from  7 Zk.  This  corresponds 
to  the  h  function  defined  in  Definition  2.1.  The  continuous 
post-operator,  postc,  defines  the  reach  sets  for  the  continuous 
states  from  7Zk  after  an  arbitrary  amount  of  time  is  elapsed. 
This  corresponds  to  F  in  Definition  2.1. 


'Kfo(x0)=  U  eAtx° ■  (16) 
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Fig.  7.  Buck  converter  prototype  controlled  with  dSPACE  DS1 103. 

An  approximated  computation  of  (-) k  is  given  in  [29]  for 
the  kth  time  step.  Hence,  a  sequence  of  convex  continuous 
sets  Bo.  0  | .  ....0  v_  i  is  computed  to  form  a  flowpipe  that 
covers  the  reach  sets  up  to  a  pre-dehned  time  such  that  N 
represents  the  number  of  time  steps.  This  flowpipe  is  then 
used  to  compute  the  transition  successors.  Only  those  states 
can  take  the  transition  that  satisfy  the  guard  associated  with 
the  present  topology  and  the  invariant  of  the  target  topology. 
This  process  is  continued  until  a  fixed  point  is  reached,  i.e., 
if  all  the  reach  sets  that  are  computed  in  the  present  iteration, 
are  contained  in  reach  sets  computed  in  the  previous  iteration, 
i.e.,  TZk+i  C  TZk-  This  signifies  that  no  new  reach  sets  could 
be  found  and  the  computation  process  may  be  terminated. 

VI.  Case  Studies 

An  experimental  setup  of  a  buck  converter,  controlled  with 
a  dSpace  DS1103  unit,  has  been  prototyped,  as  shown  in  Fig. 
7.  The  experimental  results  are  used  for  benchmarking  purpo¬ 
ses  against  MATLAB/PLECS  [48],  Simulink/Stateflow  [49], 
Monte  Carlo  simulations,  and  SpaceEx  reachability  analysis. 
Circuit  parameters  L  =  2.65  mH,  C  =  2.2  mF,  and  R  =  10 
Q  are  used  throughout  this  study.  The  non-determinism  due 
to  the  parameter  variations  is  modeled  using  the  interval 
matrices  in  SpaceEx  model.  For  a  coherent  comparison  in 
terms  of  parameter  variations  in  R,  L,  and  C,  we  have 
used  15%  tolerance  for  Monte  Carlo  simulations  and  Spa¬ 
ceEx  reachability  analysis.  We  have  used  the  Hybrid  Source 
Transformer  (HyST)  which  is  a  source-to-source  conversion 
tool  for  hybrid  automata  models  [50].  The  hybrid  automaton 
model  is  developed  using  the  java  interface  in  MATLAB,  and 
transformed  into  a  SpaceEx  compatible  model  using  HyST 
data  structures.  We  use  the  conformance  degree  to  validate 
the  hybrid  automaton  model  against  the  experimental  data. 
Then,  the  reachability  analysis  results  are  provided  for  formal 
verification  of  an  open-loop  and  a  hysteresis-controlled  buck 
converter. 

A.  Model  Validation  Using  Conformance  Degree  Testing 

We  use  notations  To  and  Iq  for  hardware  prototypes 
in  open-loop  and  closed-loop  configurations,  respectively. 
PLECS  and  Stateflow  models  are  denoted  by  'Ho p,  Hop  and 


Config. 

Type  of  Output  Trajectories 

rc  Value  (s) 

e  Value 

A  Value 

%L  -  PLECS  vs  Experiment 

3  x  icr4 

5.1515  A 

4.5570  A 

a- 

iL  -  Stateflow  vs  Experiment 

3  x  icr4 

5.0008  A 

4.5570  A 

0 

hJ 

%L  -  Stateflow  vs  PLECS 

3  x  icr4 

0.1785  A 

0  A 

c 

<D 

Cu 

vc  -  PLECS  vs  Experiment 

3  x  icr4 

1.8945  V 

1.7202  V 

vc  -  Stateflow  vs  Experiment 

3  x  10"4 

2.3201  V 

1.7202  V 

vc  -  Stateflow  vs  PLECS 

3  x  icr4 

0.6666  V 

0  V 

%L  -  PLECS  vs  Experiment 

8  x  icr4 

3.6667  A 

3.0590  A 

a- 

iL  -  Stateflow  vs  Experiment 

8  x  10“4 

3.6643  A 

3.0590  A 

0 

iL  -  Stateflow  vs  PLECS 

8  x  10~4 

0.0878  A 

0  A 

-a 

<D 

vc  -  PLECS  vs  Experiment 

8  x  icr4 

2.8014  V 

1.5905  V 

U 

vc  -  Stateflow  vs  Experiment 

8  x  10”4 

2.7677  V 

1.5905  V 

vc  -  Stateflow  vs  PLECS 

8  x  10”4 

0.0580  V 

0  V 

TLos ,  TLcs,  respectively,  where  subscript  O  denotes  an  open- 
loop  and  C  denotes  a  closed-loop  configuration.  The  computed 
s  values  against  rc  (as  defined  in  Section  III)  are  tabulated 
in  Table  I  for  the  corresponding  output  trajectories.  It  is  evident 
from  Table  I  that  the  e  values  of  Ho p  and  TLos  as  well  as 
TLcp  and  Tics  are  close  enough  (also,  as  seen  in  Fig.  8). 
We  have  computed  conformance  degrees  for  the  prototype 
buck  converters,  i.e.,  To  and  Iq,  in  comparison  with  other 
models,  i.e.,  TLop,  TLos  and  TLcp ,  Tics-  We  also  define  the 
absolute  value  of  the  maximum  difference  measured  between 
the  two  given  output  trajectories  as  A  for  a  given  time  duration 
tc.  The  measured  A  values  are  tabulated  in  Table  I.  The  e 
values  depicted  in  Table  I  provide  enough  wiggle  room  in 
comparison  with  the  corresponding  A  to  validate  that  TLop 
and  TLos  are  reasonable  abstractions  for  I q,  whereas  TLcp 
and  Hcs  are  reasonable  abstractions  for  2 c-  Consider,  for 
example,  the  case  of  a  closed-loop  buck  converter.  The  1st 
row  under  closed-loop  configuration  in  Table  I  provides  the  e 
value  (i.e.,  e  =  3.6667  A)  and  A  value  (i.e.,  A  =  3.0590  A ), 
as  we  compare  the  inductor  current  (ip)  output  trajectories 
for  PLECS  and  experimental  prototype.  A  of  the  two  output 
trajectories  remain  within  e  (as  also  depicted  in  Fig.  9  (a)). 
This  is  also  true  for  the  corresponding  output  trajectories 
of  capacitor  voltage  ( vc )■  Accordingly,  the  hybrid  automata 
models  are  validated  in  conformance  with  both  the  open-loop 
and  the  closed-loop  converter  prototypes. 

B.  Formal  Verification  of  the  Open-loop  Buck  Converter 
We  consider  the  voltage  stability  specification  to  perform 
formal  verification.  For  example,  for  ts  =  0.025  s,  and 
Vref  =  48  V,  we  define  7  =  6  V.  This  results  in  an 
upper  voltage  bound  of  54  V,  and  lower  voltage  bound  of 
42  V,  as  shown  in  Fig.  8(b)  by  dotted  lines.  The  input 
parameters  are  Vtn  =  100  V,  and  fs  =  60  kHz.  The 
output  trajectories  and  phase-plane  responses  are  considered 
for  the  startup  transients  of  the  open-loop  buck  converter. 
The  parameters’  variations  have  been  modeled  using  interval 
analysis  in  SpaceEx  model,  and  also  included  in  the  Monte 
Carlo  simulation.  The  reachability  analysis  results,  obtained 
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Fig.  8.  Startup  transients  for  an  open-loop  buck  converter  using  interval  matrices  including  Stateflow,  PLECS,  experiment,  Monte  Carlo,  and 
SpaceEx;  (a)  current  vs.  time,  (b)  voltage  vs.  time,  and  (c)  phase  portrait. 


using  SpaceEx,  are  plotted  in  Fig.  8.  It  can  be  seen  that  the 
steady-state  inductor  current  and  capacitor  voltage  waveforms 
lie  within  the  reachability  analysis  results,  i.e.,  the  simulations 
and  measurement  data  are  contained  within  the  reach  sets. 
Moreover,  we  verify  that  Veit)  £  [42,54]  for  t  >  ts  for 
Stateflow,  PLECS,  measurement  data,  Monte  Carlo  analysis, 
and  SpaceEx  analysis  results. 

C.  Verification  of  the  Hysteresis-controlled  Converter 

We  define  the  voltage  stability  specification  for  the  closed- 
loop  buck  converter  to  perform  formal  verification.  For  ts  = 
0.012  s,  and  Vref  =  12  V,  we  define  7  =  1  V.  This 
leads  to  upper  and  lower  voltage  bounds  of  13  and  11  V, 
respectively,  as  shown  by  dotted  lines  in  Fig.  9(b).  In  this  case 
study,  the  time-dependent  and  the  time-independent  models 
(as  mentioned  in  Section  II)  are  considered.  First,  SpaceEx 
reachability  analysis  is  performed  for  the  time-dependent 
model.  The  new  parameters  are  Vln  =  24  V,  Vref  =  12  V, 
and  fs  =  50  kHz.  The  trajectories  are  shown  in  Fig.  9  for 
Stateflow,  PLECS,  and  experimental  data  along  with  reach  sets 
computed  using  SpaceEx.  The  Stateflow,  PLECS,  and  SpaceEx 
results  match  right  from  the  start  until  the  steady  state  is 
reached.  Experimental  results  match  that  of  Stateflow,  PLECS, 
and  SpaceEx  in  the  steady  state.  It  can  be  observed  in  Fig.  9 
that  Stateflow,  PLECS,  and  measured  results  remain  within  the 
reach  sets  computed  using  SpaceEx,  verifying  vc(t)  £  [11, 13] 
for  t  >ts. 

We  can  formally  verify  the  time-independent  SpaceEx  mo¬ 
del  for  an  unbounded  time,  i.e.,  t  — >  00,  by  excluding  r.  This 
would  not  be  possible  through  Monte  Carlo  analysis  as,  even 
for  a  limited  time  span,  one  has  to  take  into  account  infi¬ 
nite  number  of  possible  combinations.  We  have  successfully 
achieved  a  fixed  point  using  SpaceEx,  with  unbounded  time, 
and  with  all  possible  parameter  variations.  The  phase-plane 
plots  are  given  for  the  start-up  transients  in  Fig.  10.  As  seen, 
all  results  remain  within  the  computed  reach  sets  as  t  — >  00, 
verifying  vc(t)  £  [11, 13]  as  t  — »  00. 

A  comparison  of  Monte  Carlo  analysis  and  SpaceEx  re¬ 
achability  analysis,  in  term  of  computation  times,  is  shown 
in  Table  H.  Both  are  run  on  a  Windows  7  SP1  (64  bit) 
platform,  with  Intel  (R)  core  i7-2600  CPU  with  3.40  GHz,  16.0 


version  3.7.3,  and  SpaceEx  version  0.9. 8d.  While  infinite  ite¬ 
rations  are  required  to  have  full  confidence  in  model  validation 
through  Monte  Carlo  analysis,  we  have  only  used  finite  (i.e., 
2000)  iterations  as  would  be  done  in  practice.  Even  then,  it  is 
evident  that  the  SpaceEx  reachability  outperforms  the  Monte 
Carlo  analysis  in  computation  time,  as  seen  in  Table  II. 

VII.  Conclusion 

A  hybrid  automaton  modeling  approach  for  PWM  DC- 
DC  converters  is  developed.  We  have  used  the  conformance 
testing  for  model  validation  when  compared  with  a  hardware 
prototype  of  DC-DC  converters.  The  interval  matrices  analysis 
accommodates  the  model  non-determinism  caused  by  variati¬ 
ons  in  component  values.  Reachability  analysis  frameworks 
are  developed  for  formal  verification  of  the  resulting  hybrid 
automata  models.  It  is  shown  that  the  proposed  reachability 
analysis  outperforms  the  brute  force  Monte  Carlo  analysis  in 
computation  time  and  confidence  level. 
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Fig.  9.  Time-dependent  hysteresis-controlled  converter  analysis  using  interval  matrices  including  Stateflow,  PLECS,  experiment,  Monte  Carlo,  and 
SpaceEx;  (a)  current  vs.  time,  (b)  voltage  vs.  time,  and  (c)  phase  portrait. 


Fig.  10.  Time-independent  hysteresis-controlled  converter  analysis 
using  interval  matrices:  Stateflow,  PLECS,  experiment,  Monte  Carlo, 
and  SpaceEx. 
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Abstract — Power  electronics-intensive  DC  microgrids  use  in¬ 
creasingly  complex  software-based  controllers  and  communica¬ 
tion  networks.  They  are  evolving  into  cyber-physical  systems 
(CPS)  with  sophisticated  interactions  between  physical  and  com¬ 
putational  processes,  making  them  vulnerable  to  cyber  attacks. 
This  work  presents  a  framework  to  detect  possible  false-data 
injection  attacks  (FDIA)  in  cyber-physical  DC  microgrids.  The 
detection  problem  is  formalized  as  identifying  a  change  in 
sets  of  inferred  candidate  invariants.  Invariants  are  microgrids 
properties  that  do  not  change  over  time.  Both  the  physical 
plant  and  the  software  controller  of  CPS  can  be  described 
as  Simulink/Stateflow  (SLSF)  diagrams.  The  dynamic  analysis 
infers  the  candidate  invariants  over  the  input/output  variables  of 
SLSF  components.  The  reachability  analysis  generates  the  sets 
of  reachable  states  (reach  sets)  for  the  CPS  modeled  as  hybrid 
automata.  The  candidate  invariants  that  contain  the  reach  sets 
are  called  the  actual  invariants.  The  candidate  invariants  are  then 
compared  with  the  actual  invariants,  and  any  mismatch  indicates 
the  presence  of  FDIA.  To  evaluate  the  proposed  methodology, 
the  hybrid  automaton  of  a  DC  microgrid,  with  a  distributed 
cooperative  control  scheme,  is  presented.  The  reachability  anal¬ 
ysis  is  performed  to  obtain  the  reach  sets  and,  hence,  the 
actual  invariants.  Moreover,  a  prototype  tool,  FIYbrid  iNvariant 
GEneratoR  (Hynger),  is  extended  to  instrument  SLSF  models, 
obtain  candidate  invariants,  and  identify  FDIA. 

Index  Terms — Cyber-physical  systems,  dc  microgrid,  dis¬ 
tributed  control,  false-data  injection  attack,  hybrid  automaton. 


I.  Introduction 

ISLANDED  multi-converter  DC  microgrids  have  advan¬ 
tages  over  their  AC  counterparts,  including  higher  reli¬ 
ability,  simpler  control,  and  more  efficient  interfacing  with 
naturally-DC  renewable  energy  sources,  electronics  loads, 
and  energy  storage  units  [1],  [2].  Therefore,  DC  microgrids 
have  emerged  as  a  key  technology  for  the  future,  and  their 
related  control  methodologies  are  also  evolving.  Given  the 
well-established  advantages  of  distributed  control  schemes 
over  centralized  control  methodologies,  the  migration  from 
current  central  controllers  to  future  distributed  schemes  is  in¬ 
evitable  [3]— [8].  The  centralized  control  systems  require  two- 
way,  high  bandwidth  communication  links  between  the  central 
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controller  and  every  other  agent,  and  expose  a  single  point- 
of-failure.  Moreover,  sparsity  of  communication  networks  uti¬ 
lized  in  distributed  control  schemes  reduces  the  infrastructure 
cost,  and  improves  solution  scalability  compared  to  a  fully- 
connected  communication  network. 

These  DC  microgrids  are  evolving  into  cyber-physical  sys¬ 
tems  (CPS)  with  sophisticated  software-based  control  and 
communication  networks.  Such  CPS  are,  however,  vulnerable 
to  cyber  attacks,  as  there  is  no  central  entity  to  monitor 
activities  of  all  DC-DC  converters  leading  to  a  limited  global 
situational  awareness.  This  vulnerability  is  analogous  to  the 
situation  in  cyber-physical  power  systems  that  have  faced 
various  types  of  cyber  attacks,  e.g.,  false-data  injection  attack 
(FDIA)  [9],  denial  of  service  [10],  [11],  jamming  [12],  and 
random  attacks  [13].  Some  prevention  strategies  for  jamming 
include  frequency  hopping,  direct-sequence  spread  spectrum 
technique,  channel  surfing,  and  protocol  hopping  [14].  In  this 
work,  detection  of  FDIA  in  power  electroncis-intensive  DC 
microgrids  is  considered  that  involves  spoofing  a  signal,  either 
in  sensors  or  the  communication  network,  through  an  attack 
vector  that  aims  to  disrupt  the  steady-state  operation  [9]. 
The  attack  vector  formulation  is  a  sophisticated  process,  and 
requires  expert  knowledge  of  the  entire  system.  The  intruder 
should  have  either  physical  access  to  a  specified  number  of 
meters,  or  a  complete  knowledge  of  the  infrastructure  and  the 
communication  network  [9]. 

The  preventive  measures  against  FDIA  include  physical 
security,  information  security,  and  communication  security. 
With  regards  to  the  physical  security,  a  minimum  number 
of  strategically  selected  set  of  sensor  measurements  (called 
as  basic  measurements)  that  need  to  be  protected  to  thwart 
FDIA  has  been  proposed  [15].  Moreover,  phasor  measurement 
units  (PMUs)  can  be  strategically  placed  to  protect  power  grids 
against  such  attacks  [16].  However,  PMUs  are  also  vulnerable 
due  to  their  use  of  global  positioning  systems  [17],  With 
regards  to  information  security,  a  prevention  strategy  against 
FDIA  involves  dynamically  changing  the  information  structure 
of  microgrids  [18],  In  general,  the  communication  security 
can  be  improved  using  stringent  cryptographic  techniques,  i.e., 
encryption,  authentication,  and  key  management  for  power 
systems  [19].  For  example,  a  communication  security  architec¬ 
ture  for  distributed  microgrid  control  [20]  exchanges  encrypted 
information.  A  trusted  sensing  base  is  proposed  in  the  form 
of  a  current  transformer  that  encrypts  the  AC  power  signal 
before  sending  it  to  PMUs  [21]. 

Recent  work  on  FDIA  detection,  albeit  in  power  sys¬ 
tems  [11],  [13],  [22]-[29],  broadly  employs  state  estima¬ 
tion  processes,  e.g.,  using  Kalman  filters  [13],  sparse  op- 
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Fig.  1.  The  proposed  FDIA  detection  framework  bridges  the  gap  between  the 
software-based  anomaly  detection  techniques  and  power  electronics-intensive 
DC  microgrids  modeled  as  hybrid  automata. 


Fig.  2.  Hybrid  automaton  of  a  cyber-physical  DC  microgrid  showing  converter 
and  controller  interactions.  Each  converter,  its  corresponding  controller,  and  its 
communication  links  are,  altogether,  considered  as  an  agent.  Each  agent  shares 
its  information  with  the  neighboring  agents  on  the  communication  graph. 


timization  [22],  generalized  likelihood  ratio  [23],  Kullback- 
Leibler  distance  [24],  Chi-square  detector  and  similarity 
matching  [25],  state  forecasting  [26],  and  machine-learning 
techniques  [27].  However,  to  the  best  of  authors’  knowledge, 
FDIA  detection  in  software-intensive  DC  microgrids  is  not 
systematically  studied  yet.  This  work  aims  to  formalize  the 
FDIA  detection  problem  as  a  change  in  sets  of  inferred 
invariants',  system  properties  that  do  not  change  over  time. 
Here,  invariants  are  defined  in  terms  of  bounds  over  the  output 
voltage  and  current  of  individual  converters. 

The  overall  block  diagram  of  the  proposed  FDIA  detection 
framework  is  shown  in  Fig.  1.  The  candidate  invariants  are 
inferred  from  the  Simulink/Stateflow  (SLSF)  model  of  the 
DC  microgrid.  Hynger  (HYbrid  iNvariant  GEneratoR)  [30] 
tool  is  used  to  provide  an  interface  between  the  SLSF  model 
and  the  Daikon  tool  [31],  [32],  Daikon  is  a  software-based 
invariant  inference  tool.  Hynger  takes  the  SLSF  model  as 
an  input,  executes  it  to  generate  time  traces,  and  transforms 
them  into  a  format  compatible  with  Daikon  to  generate  candi¬ 
date  invariants.  Moreover,  the  cyber-physical  DC  microgrid 
is  formally  modeled  as  multi-agent  hybrid  automata,  and 
the  reachability  analysis  is  performed  using  SpaceEx  [33] 
to  obtain  the  reachable  set  of  states  (called  the  reach  sets). 
The  Hynger/Daikon  combination  provides  only  the  candidate 
invariants.  The  SpaceEx  tool  is  used  concurrently  in  the  pro¬ 
posed  framework  to  obtain  the  actual  invariants.  The  candidate 
invariants  that  contain  the  reach  sets  are  called  the  actual 
invariants.  The  candidate  invariants  are  then  compared  with  the 
actual  invariants,  and  any  mismatch  indicates  the  presence  of 
FDIA.  A  mitigation  strategy  can  then  disconnect  the  affected 
converter  and  prevent  the  microgrid’s  instability. 

The  remainder  of  this  paper  is  organized  as  follows:  The 
hybrid  automaton  modeling  of  DC  microgrids  that  includes 
both  physical  and  cyber  layers  is  discussed  in  Section  II.  The 
FDIA  detection  framework  for  DC  microgrids  is  discussed 
in  Section  III.  Section  IV  studies  a  DC  microgrid  prototype, 
with  an  analysis  of  FDIA  effects,  detection  using  the  proposed 
framework,  and  mitigation.  Section  V  concludes  the  paper. 


II.  Cyber-physical  DC  Microgrid  as  Multi-agent 
Hybrid  Automata 

The  proposed  FDIA  detection  framework  requires  the  CPS 
modeled  as  SLSF  diagrams  and  as  hybrid  automata  to  obtain 
the  candidate  invariants  and  the  reach  sets,  respectively.  A 
hybrid  automaton  [34]  is  a  formal  model,  essentially  a  finite- 
state  machine  with  additional  continuous  dynamic  variables. 
Cyber-physical  DC  microgrids  can  be  modeled  as  multi-agent 
hybrid  automata,  where  power  electronics  DC-DC  converters 
(referred  to  as  converters)  form  the  physical  layer,  and  the 
software-based  controller  with  the  communication  network 
among  converters,  altogether,  form  the  cyber  layer.  Each  con¬ 
verter,  with  its  corresponding  controller  and  communication 
links,  is  considered  an  agent,  and  its  hybrid  automaton  is 
shown  in  Fig.  2.  This  hybrid  automaton  exchanges  information 
with  its  two  immediate  neighbors,  e.g.,  ( i  +  l)th  and  (i  —  l)th 
agents  in  Fig.  2,  through  global  variables  to  implement  a 
cooperative  control  protocol. 

A.  Modeling  the  Physical  Layer 

The  output  voltage  v°ut  and  output  current  of  the  ith 
converter  are  regulated  by  controlling  the  MOSFET  switch 
through  the  corresponding  control  layer.  The  switching  state  of 
the  MOSFET  switch  leads  three  different  topologies  (switch¬ 
ing  sub-interval)  as  shown  in  Fig.  2.  The  state  of  a  hybrid 
automaton  may  change  either  through  a  continuous  flow  tra¬ 
jectory  within  a  given  topology,  or  through  a  discrete  transition 
between  two  given  topologies. 

1)  Formal  Hybrid  Automaton:  Let  1R”  be  the  set  of  n- 
dimensional  reals,  and  2X  be  the  power  set  of  a  given  set 
X ,  i.e.,  the  set  of  all  the  subsets  of  X. 

Definition  2.1:  A  hybrid  automaton  is  defined  by  a  tuple 
U  =  (Q,X,<d,U,F,T,E,G,inv): 

•  Q  =  {<Zr,  72,  ■•■■7  Qn}  is  a  finite  set  of  topologies. 

•  X  is  a  finite  set  of  continuous  variables,  with  Vie 
X  3  val(x)  £  R,  where  val(x)  is  a  valuation  of  a;  as  a 
result  of  a  function  mapping.  X  =  Xg  IJ  A ; ,  such  that  Xg 
is  the  set  of  global  variables  and  Xi  is  the  set  of  local 
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variables.  Further,  Xg  =  In  U  O,  where  In  is  the  set  of 
global  input  variables  and  O  is  the  set  of  global  output 
variables.  A  state  is  defined  by  s  =  ( q ,  val(x))  £  Q  x  R. 
0  C  Q  x  IRn  is  a  set  of  initial  conditions. 

U  =  {ui,U2,  ...hat}  is  the  set  of  inputs  for  each  topology. 
F  is  a  finite  set  of  ODEs  defined  for  each  q  £  Q  over 
the  continuous  variables  x  £  X.  F(q,x)  defines  the 
continuous  dynamics  for  each  q  £  Q  over  a  time  period 
T,  and  assigns  a  Lipschitz  continuous  vector  space  in  R". 
T  is  a  finite  set  of  continuous  flow  trajectories  that  define 
val( x)  over  [0  T ]  from  given  initial  conditions  (q,x o)  £ 
0,  such  that  V  r(q,x)  £  T,  3  s  £  r(q,x)  that  satisfies 
inv(q)  (i.e.,  s  £  r(q,x)  t=  inv(q)).  A  continuous  flow 
trajectory  is  given  by 


U  =  {[ujn,  0,0,0]',  [0,0, 0,0]',  [0,0, 0,0]'}  forms  the 
input  vector  set. 

E  =  {(qi,q2,gi2,x')  ,(q2,qi,g2i,x')  ,(q2,q3,g23,x') , 
(fei  <7i,  ff3i> x')}  defines  the  feasible  discrete  transitions, 
e.g.,  ((72,  <73,  g23,x')  means  that  a  discrete  transition  from 
topology  <72  to  q3  is  allowed,  if  the  guard  (723  =  {(if  < 
0)}  is  satisfied  and  the  continuous  state  is  reset  to  x'. 
Guard  set,  for  the  corresponding  elements  of  E,  is  defined 
by  G  =  {( control i  ==  0) ,  ( control i  ==  1) ,  (if  <  0)  , 
( control  ==  1)}.  Signals  received  from  the  control 
layer  are  controli  ==  1  and  controli  ==  0  to  set  the 
MOSFET  ON  and  OFF,  respectively. 

The  continuous  flow  trajectory  is  defined  by  (2),  with 
the  corresponding  state  matrices  for  each  topology. 


T~(q,x)  =  x0  +  J  F(q,x)dt.  (1) 

o 

•  E  is  a  finite  set  of  feasible  discrete  transitions  allowed 
among  the  topologies.  It  is  defined  by  a  tuple  e  = 
{q,q'  ,g,x'),  such  that  a  discrete  transition  is  allowed 
from  source  topology  q  to  the  destination  topology  q'  only 
when  the  associated  guard  condition  g  is  satisfied,  and  the 
continuous  state  is  updated  to  x'  after  the  transition.  It 
might  not  be  possible  to  visit  the  entire  set  of  topologies 
from  one  particular  topology. 

•  G  C  2a  is  the  guard  set  such  that  V  e  3  g  £  G.  A  guard 
must  be  satisfied  by  a  state  to  take  a  discrete  transition 
from  a  given  topology  to  another.  A  state  s  =  ( qk ,  val(x)) 
satisfies  g  (i.e,  s\=  g)  iff  qk  =  qx  £  e  =  (. quq[,  g,  x')  and 
val{ x)  £  g. 

•  inv  is  a  finite  set  of  invariants,  where  an  invariant  is  as¬ 
sociated  to  each  given  topology,  i.e.,  V  q  £  Q  3  inv(q)  C 
R".  An  invariant  is  a  property  of  the  hybrid  automaton 
that  must  be  satisfied  by  all  the  states  for  a  given  topology. 
A  state  s  \=  inv(q)  iff  val{x)  £  inv(q). 

If  a  state  (q,val(x))  does  not  satisfy  an  invariant  inv(q), 
the  continuous  state  x  stops  evolving  within  a  topology.  The 
guard  function  ensures  a  discrete  transition  to  an  appropriate 
topology  once  the  corresponding  guard  is  satisfied.  Here, 
invariants  and  guards  are  defined  in  the  form  of  bounds 
over  continuous  state  variables.  The  semantics  of  the  hybrid 
automaton  77  is  defined  by  its  execution,  e.  An  execution  is 
defined  as  a  sequence  of  states,  e  =  s0>  Si,  S2, . . . ,  obtained  as 
a  result  of  continuous  flow  trajectories  and  discrete  transitions. 

2)  Instantiation  of  the  Physical  Layer:  The  hybrid  automa¬ 
ton  of  the  ?'th  buck  converter  is  considered  for  instantiation, 
where  v"1  is  the  DC  input  voltage.  The  continuous  dynamics, 
for  a  given  topology,  is  given  by  a  set  of  state-space  equations 

dx 


dt 


—  A„X  +  Br 


U. 


(2) 


X  =  {it 

and  Xg  =  {i° 


c  tout  „out  control^'  where  Xi  =  {if, 
,  controli}. 


h 

out 


>vi 

, out 


} 


The  evolution  of  the  hybrid  automaton  model  starts  with 
initial  conditions  from  the  set  init ,  e.g.,  (<71,  £0)  G  Anii  for  a 
given  input  u\  =  [vP1,  0, 0, 0]'  and,  subsequently,  the  continu¬ 
ous  state  evolves  according  to  the  flow  function.  The  topology 
remains  the  same,  i.e.,  q  (7)  =  qk,  as  Xo  evolves  inside  the 
invariant  inv  (qi)  and  attains  a  final  value  x'  £  inv(qi). 
Once  the  continuous  state  x'  satisfies  the  corresponding  guard, 
g\2  =  {{controli  ==  0)}  corresponding  to  the  topology  <71, 
the  topology  may  transition  from  q\  to  (72,  and  the  continuous 
state  is  reset  with  a  new  value  x"  in  the  new  invariant  set 
inv  ((72)  with  a  new  input  112  =  [0, 0,  0,  0]'. 

B.  Modeling  the  Cyber  Layer 

Microgrid  control  hierarchy  is  divided  into  three  levels, 
i.e.,  primary,  secondary,  and  tertiary  [35].  Primary  control 
features  the  fastest  response,  and  is  based  entirely  on  local 
measurements  with  no  communication.  Secondary  control 
operates  on  a  slower  time  scale,  often  with  reduced  commu¬ 
nication  bandwidth  by  using  sampled  measurements.  In  this 
work,  we  consdier  two  control  objectives:  proportional  load 
sharing  among  converters,  according  to  their  power  ratings, 
and  global  voltage  regulation  of  the  distribution  bus.  These 
objectives  are  implemented  in  the  secondary  control  layer 
through  proportional  load  sharing  sub-layer  and  global  voltage 
regulation  sub-layer  (which  includes  a  voltage  observer  and 
a  noise  cancellation  module),  as  shown  in  Fig.  3.  We  use  a 
distributed  cooperative  control  scheme,  i.e.,  the  output  of  a 
particular  agent  depends  only  on  its  information  and  its  N, 
neighbors  on  the  communication  graph  [3],  A  graph  G  is 
defined  as  a  pair  (tuple)  of  a  set  of  vertices  and  edges,  i.e., 

G  =  (A, £■).  Fet  A  =  {Ai,A2, . ,  A n}  define  a  set  of  N 

vertices  (nodes),  and  e  C  A  x  A  a  set  of  edges.  An  edge  from 
node  Xi  to  A  j  is  a  pair  (Xt.  Xj)  £  e.  The  graph  is  said  to  be 


bi-directional  if  (Aj,Aj)  €  e 


(Xj,Xi)  £  e,  V  i,j  £  A. 


Aq  £  Rraxn  and  Bq  £  Rraxm  are  system  matrices.  Subscript 
q  denotes  the  appropriate  topology.  The  instantiation  of  the 
hybrid  automaton  for  the  7th  agent,  as  per  Definition  2.1,  is 
•  Three  topologies,  shown  in  Fig.  2,  are  denoted  by  Q  = 
{<7i,  <72,73 }■ 


A  graph  may  be  represented  by  an  adjacency  matrix  A  = 
[i ctij ]  with  weights  anj  >  0  if  (Aj,  A,)  £  e,  and  a.y  =  0 
otherwise.  The  local  control  protocol,  Ui  for  each  agent  i  is 


=  £ 

jeNi 


ij(xj  Xi), 


(3) 


such  that  the  control  of  each  agent  depends  only  on  the 
difference  between  its  state  and  those  of  its  neighbors.  This 
protocol  ensures  that  all  agents  reach  a  consensus. 
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Fig.  3.  Structure  of  the  cyber-physical  DC  microgrid  showing  the  cyber  and 
physical  layers  and  the  control  sub-layers. 

The  global  voltage  reference  for  N  agents  is  defined 
as  Vref  =  [v[e^ ,  v™* , ...,  vr^]T ,  input  DC  voltage  as 
Vin  =  [v™,  v™, u™]T,  output  DC  voltage  vector  as 
Vout  =  [viut,v%ut,...,v<ftlt]T,  output  current  vector  as  Iout  = 
[i°ut,  i%ut, ...,  the  voltage  estimation  vector  for  the 

voltage  observer  module  as  Vest  =  [vfst ,  V2St,  ■■■,v^t]T ,  the 
per-unit  current  vector  as  Xpu  =  x^11,  ■■■,  ]T,  and 

the  estimate  of  the  voltage  deviation  vector  for  the  noise 
cancellation  module  as  West  =  [wfst,  w%st, ...,  w^l]T .  Here, 
x?u  refers  to  the  loading  percentage  of  the  zth  agent.  As  shown 
in  Fig.  3,  Xt  depicts  the  information  vector  communicated 
from  the  zth  agent  to  the  (z— l)th  and  (i+l)th  agents,  such  that 
=  [x^u,  vfst,  wfst]T.  Moreover,  X^x,  and  Xi+1  of  Fig.  3 
are  defined  similarly.  Communication  links  are  modeled  as 
low -pass  filters  (Ti  and  Tr2  in  Appendix)  to  emulate  delays 
inherent  in  the  data  exchange  process,  as  in  [3],  [36],  [37]. 
Here,  i°ut  and  v°ut  are  passed  through  Ti  to  get  the  per-unit 
current  xLu  and  the  voltage  y V°,  respectively. 

The  control  sub-layers  are  discussed  next. 

1)  Proportional  load  sharing  sub-layer:  The  ith  agent 
shares  per-unit  current  information  with  its  immediate  neigh¬ 
bors,  i.e.,  (z  —  l)th  and  (z  +  l)th  agents.  This  sub-layer  has 
a  PI  controller  with  parameters  P(z,z, )  and  I (1. 1),  where  P 
and  /  are  N  x  N  matrices  that  contain  the  proportional  and 
integral  terms,  respectively.  If  C  is  the  adjacency  matrix  for  the 
cooperative  control  strategy,  the  per-unit  current  information 
from  (z  —  l)th  and  (i  +  l)th  agents  communicated  to  the  PI 
controller  is  processed  as 

(4) 

and 

=  (xi+ 1  -  xD  -c(l  *  + !).  (5) 

respectively.  Let  the  state  variable  of  the  PI  controller  be  x\, 
then  the  corresponding  ODE  is  given  by 

xl  =  (xi-i^i  +  a;i+i_,i-)  (6) 

The  output,  v\,  of  this  layer  is  given  by 


which  is  passed  to  the  primary  control  sub-layer. 

2)  Global  voltage  regulation  sub-layer:  If  A  is  the  adja¬ 
cency  matrix  for  the  cooperative  control  strategy,  the  voltage 
estimation  information  from  (i  —  l)th  and  (z  +  l)th  agents  is 
further  processed  as 

=  (8) 

and 

=  +  (9) 

respectively.  This  voltage  estimate  is  then  passed  through  an 
integrator,  with  the  state  variable  vfsti,  such  that 

vrti  =  (vZt1_n  +  vf?1_+i).  (10) 

In  the  noise-cancellation  module,  the  zth  agent  shares  the 
estimate  information  of  the  voltage  deviation  wfst  with  its 
immediate  neighbors.  The  actual  voltage  deviation  for  the  ith 
agent  is 

Wi  =  (vtst  -  O  ■  (ID 

If  B  is  the  adjacency  matrix  for  the  cooperative  control 
strategy,  the  information  about  the  estimate  of  the  voltage 
deviation  from  (i  —  l)th  and  (i  +  l)th  agents  is 

1-H  =  K-1  -  <st)  -B(i,  i  -  1)  (12) 

and 

<ft^  =  -  <st)  .B(i,  *  +  1),  (13) 

respectively.  This  estimate  is  passed  through  an  integrator,  with 
the  state  variable  wfsU,  such  that 

wru  =  {<- u +<£_*)•  (i4) 

The  estimate  for  the  voltage  deviation,  u^st,  is 

wtst  =  Wi  +  wfsti.  (15) 

This  estimate  is  then  passed  to  a  second  integrator  with  a  gain 
K  of  dimension  N  x  N,  and  with  the  state  variable  iufstl'‘, 

^estii  =  west  (16) 

The  average  voltage  of  the  microgrid  as  estimated  by  the  ith 
agent,  based  on  the  neighbor  information,  is 

vT 9  =  v\st  =  v\sti  +  v°ut  -  wtstii.K{i,  i).  (17) 

This  sub-layer  has  a  PI  controller  with  parameters  P(z,  z, )  and 
The  difference  between  the  global  reference  voltage 
and  the  global  average  voltage  as  determined  by  the  ith  agent 
is  passed  through  this  PI  controller.  Let  the  state  variable  for 
PI  controller  be  denoted  by  v"V!P ,  then  the  ODE  is  given  by 

v“V3i  =  «e/  -  v“V9).I(i,i).  (18) 

The  voltage  regulation  term  at  the  controller  output  is 

vfe 9  =  vT3i  +  Ke/  -  v™9).p{i,  i).  (19) 

3)  Primary  control  sub-layer:  There  is  a  PI  controller  with 
parameters  Pmc  and  and  a  transfer  function  T2.  The 
output  of  T2  is  denoted  by  yj710.  The  input  m”10  is 


’A  =  (C 


1  -ti 


i  +  Xil i_n)  ■ P{i,i)  +  x\ , 


(7) 


me  ref  .  i  ,  area 
U =  V +  Vi  +  Vi  y . 


(20) 
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The  expression  for  v\  and  v\re9  are  given  by  (7)  and  (19), 
respectively.  The  ODE  for  the  state  variable  xvf  associated 
with  the  PI  controller  is  given  by 


C  =  (y.r  -  Vi°)  -Imc 

The  output  of  this  sub-layer,  t/*,  is  given  by 

yl  =  Pmc.(y7lc-yi°)  +  <, 


(21) 


(22) 


that  drives  the  MOSFET  of  the  ith  converter.  The  cyber  layer 
has  two  topologies,  i.e.,  controli_l  and  controli_0, 
as  shown  in  Fig.  2,  to  generate  control  signal,  controli. 
It  may  evaluate  to  controli  ==  1  and  controli  ==  0, 
that  correspond  to  ‘ON’  and  ‘OFF’  pulses  for  the  MOSFET, 
respectively.  The  hybrid  automaton  generates  controli  ==  1 
in  controli_l,  and  controli  ==  0  in  controli_0. 
The  ODEs  developed  for  the  cyber  layer  of  DC  microgrid 
above  are  used  to  describe  the  continuous  dynamics  for  the 
two  topologies.  The  switching  logic,  logici,  is  formulated 
using  (22),  the  elapsed  time  ti ,  and  the  time  period  T,  of 
the  ith  agent.  This  is  implemented  in  the  hybrid  automaton 
model  as  a  guard  to  enforce  the  discrete  transition  from 
topology  controli_l  to  the  topology  controli_0,  hence 
generating  control  signal  controli  ==  0.  Whereas,  transition 
from  control i_0  to  control i_l  is  entirely  dependent 
upon  the  time  period  T,  that  forms  the  corresponding  guard  to 
ensure  a  periodic  switching.  This  transition  is  enforced  by  the 
guard  ti  >  Ti,  hence  generating  control  signal  controli  ==  1. 

4)  Instantiation  of  the  cyber  layer:  The  instantiation  of  the 
hybrid  automation  model  for  the  cyber  layer  of  the  zth  agent, 
as  per  Definition  2.1,  is 

•  Two  topologies  are  denoted  by  Q  =  {<74,  <7.5 }. 

•  The  continuous  state  vector  is  I  =  X )  l.J  Xg, 

where,  Xi  =  {x\,  vfsti,  wfs‘ H,  v™9*,  xf }  ; 


= 


>  xi+ H  Ui+H  Wi+H  Xi-H  ui- 1? 


{x^,vr-,w- 
Wil\ ,  controli } . 

•  E  =  {(<74,  Q5,945,  x’)  ,  (q5, 54, 554,  x’)}  defines  the  fea¬ 
sible  discrete  transitions,  e.g.,  (tfe,  54, 554, x')  means  a 
discrete  transition  from  the  topology  r/-,  to  54  is  allowed, 
if  the  guard  g 54  =  {(ti  >  Ti)}  is  satisfied  and  the 
continuous  state  is  reset  to  a  new  value  x' . 

•  Guard  set,  for  the  corresponding  elements  of  E,  is  defined 
by  G  =  {( logici  <  0) ,  (ti  >  Ti)}. 

•  The  continuous  flow  trajectory  is  given  by  ODEs  in  (6), 
(10),  (14),  (16),  (18),  and  (21)  for  both  topologies. 

The  control  layer  and  the  physical  layer  both  interact  with  each 
other  and  exchange  controli  and  X°ut  as  shown  in  Fig.  2, 
where  X°ut  =  [v°ut,  i°ut]T  and  controli  drives  the  switching 
in  the  physical  layer.  A  50  ps  fixed  time-step  for  the  numerical 
solver  in  the  Simulink,  and  4  /.is  sampling  time  are  used  in 
the  dSPACE  platform. 


Definition  2.2:  Fet  Hip  and  H,c  denote  the  hybrid  au¬ 
tomata  of  the  converter  and  the  controller  for  the  ith  agent, 
respectively.  They  are  compatible  if  they  meet  following  three 
conditions 

1)  I n^  X  O ic  U  U  0(i—i^p, 

2)  Inic  £  Oip  U  U  O^i—  i)c,  and 

3)  Oip  n  oic  n  0(t-Ti)c  n  n  i)c  n  0^_i^p  0. 

Subscripts  p  and  c  denote  the  plant  (i.e.,  converter)  and  the 
controller,  respectively. 

The  corresponding  input  and  output  variables,  for  the  zth  agent, 
are 

{Inip  =  {controli}, 

Inic  =  {vfut,ifut,Xi-UXi+1}, 

()ir  -  '7"! • 

Oic  =  {controli,  X,}. 

The  output  variables  for  the  ( i  +  l)th  and  (i  —  l)th  agents  are 

{/O  _  fn.out  tout 

^(i+ l)p 

^(i+i)c  {controli-{-i ) 

_  rout  tout  i  ^  ' 

\yi—  1)  1J? 

Oig- i)c  =  {controli-i,  Xi_i} . 

It  is  obvious  that  the  zth  agent  (comprising  converter  and  con¬ 
troller)  meets  the  compatibility  conditions  of  Definition  2.2, 
and  a  parallel  composition  can  be  formed.  For  the  zth  agent,  the 
parallel  composition  is  Hi  =  Hlp  ||  HiC.  The  DC  microgrid 
is  a  parallel  composition  of  N  agents,  i.e,  H\  ||  H2  ||  •  •  •  || 
Hi  ||  •  •  •  |!  Hn  ||  Hu  where  V  i  Ht  =  Hip  ||  Hic. 


III.  FDIA  Detection  Using  Hynger 
A.  FDIA  Scenario  Formulation 

In  cyber-physical  DC  microgrids,  the  information  among  the 
agents  is  shared  through  the  global  variables  (e.g.,  Xf)  that  are 
vulnerable  to  the  FDIA.  An  FDIA  mixes  the  original  data/mea¬ 
surements  vector  with  a  malicious  vector.  The  intruder  may 
target  the  global  variables  and  the  sensors  data  to  disturb  the 
consensus  procedure,  as  will  be  demonstrated  in  Section  IV. 
In  an  unconstrained  scenario,  the  intmder  has  access  to  all 
the  global  variables,  and  may  randomly  select  some  (or  all). 
Under  constrained  FDIA,  the  intruder  has  limited  access  to 
one  or  a  few  global  variables,  and  formulates  the  FDIA  vector 
to  target  these.  Fet  Xg  £  Rk  be  the  vector  containing  the 
global  variables.  FDIA  vector  W  £  1R/1'  may  be  formulated  to 
obtain  the  compromised  vector  Z  =  Xg  +  aW,  where  ct  is  a 
real  valued  multiplicative  factor  that  defines  the  weight  of  the 
FDIA  vector.  Each  element  of  the  FDIA  vector  is  denoted  by 
Wi,  such  that  a  nonzero  entry  signifies  that  the  corresponding 
global  variable  in  Xg  is  targeted.  For  unconstrained  FDIA,  all 
elements  of  IT  £  ]Rfc  are  nonzero. 


C.  Hybrid  Input/Output  Automata  Conditions 


B.  Hynger  -  An  Overview 


The  closed-loop  control  systems  are  modeled  using  hybrid 
input/output  automata  (HIOA),  to  form  as  a  singleton  hybrid 
automaton  [38],  Here,  the  converter  and  the  controller  are 
modeled  as  two  hybrid  automata,  interacting  with  each  other  in 
a  parallel  composition,  provided  that  their  local  variables  are 
disjoint  from  each  other  and  the  two  automata  are  compatible. 


Hynger  is  a  MATFAB-based  software  tool  to  produce  in¬ 
variants  for  cyber-physical  systems  modeled  using  SFSF.  Hyn¬ 
ger  uses  MATFAB’s  application  program  interfaces  (APIs) 
to  interact  with  SFSF  models  during  simulations  [30],  and 
inserts  instrumentation  points  for  selected  state  variables. 
Instrumentation  points  may  be  regarded  as  the  observation 
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r 

Infer  Candidate 

Hybrid  Automaton 

(Hynger) 

Simulate 
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Invariants 

(SpaceEx) 

(SLSF) 

(Daikon) 

Rh  I 

(a)  Open-loop  or  periodically  controlled  /  time-triggered  (with  period  A) 


(b)  Aperiodically  controlled  /  event-triggered 

Fig.  4.  The  instrumentation  points  are  added  by  Hynger  into  open-loop, 
periodically  or  aperiodically  controlled  SLSF  models. 

points  to  record  state  variable  values  at  each  simulation  time- 
step.  Hynger  can  instrument  both  open-loop  or  periodic,  and 
aperiodic  control  actions,  //,  as  shown  in  Fig.  4.  The  instru¬ 
mentation  points  are  inserted  into  the  SLSF  diagram  using 
function  calls  through  following  two  types  of  callbacks: 

1)  Precondition  Callback:  It  is  called  before  a  Simulink 
block  output  method  executes,  i.e.,  the  valuation  of  the  state 
variable  is  recorded  before  the  Simulink  block  execution. 
Hence,  the  state  valuation  is  recorded  at  time  t. 

2)  Postcondition  Callback:  It  is  called  after  a  Simulink 
block  output  method  executes,  i.e.,  the  valuation  of  a  state 
variable  is  recorded  after  the  Simulink  block  execution.  Hence, 
the  valuation  of  the  state  is  recorded  at  time  t  +  5,  where  S  is 
the  simulation  time-step. 

As  the  SLSF  diagram  is  simulated  using  Hynger,  these 
instrumentation  callback  functions  automatically  insert  the 
instrumentation  points  to  generate  a  trace  file  format  compat¬ 
ible  with  Daikon,  a  dynamic  analysis  tool  used  to  generate 
likely  invariants  for  software  programs  [31].  The  analysis 
performed  on  a  software  program  by  actually  executing  it 
on  a  host  processor  is  called  the  dynamic  analysis.  As  the 
computational  overhead  for  Hynger  grows  linearly  with  the 
number  of  monitored  state  variables  [30],  the  user  can  select 
fewer  state  variables  for  monitoring  (e.g.,  the  output  voltages 
and  currents  in  DC  microgrid)  to  reduce  the  computational 
overhead.  Moreover,  instead  of  selecting  the  entirety  of  the 
Simulink  model  for  instrumentation,  the  user  can  select  fewer 
Simulink  blocks  to  further  reduce  the  performance  overhead. 
C.  FDIA  Detection  Framework 

This  framework  involves  inferring  and  checking  sets  of 
invariants  to  determine  if  an  FDIA  is  underway.  While  this 
builds  on  the  Hynger  tool,  extensions  will  be  required  to 
execute  the  tool  and  analyze  results  at  runtime.  The  FDIA 
detection  framework  is  shown  in  Fig.  5.  A  CPS  model  is 
provided  as  an  SLSF  diagram  A.  The  SLSF  diagram  is 
instrumented  (denoted  as  A)  using  the  Hynger  tool,  and  is 
executed  to  generate  a  set  of  sampled,  finite-precision  traces 
ST  for  given  initial  condition  0  £  0.  This  adds  instrumentation 
points  for  every  input  and  output  signal  in  the  SLSF  diagram. 
These  generated  traces  are  in  Daikon  compatible  format  that 
are  passed  on  to  Daikon,  and  analyzed  to  generate  a  set  of 
candidate  invariants  <1>.  However,  Hynger/Daikon  combination 
provides  only  the  candidate  invariants  when  used  as  standalone 
invariant  generation  tool.  Each  element  ip  £  $  is  then  checked 
as  actual  invariant  using  the  reachability  analysis.  The  SpaceEx 
reachability  analysis  tool  [33]  is  used  to  obtain  the  actual 
invariants.  Changes  in  $  over  time  indicates  an  FDIA. 


CPS  Models 
(SLSF) 


Anomaly: 
Mitigate  action 


Match:  No  Action 


Fig.  5.  FDIA  detection  framework  using  Hynger/Daikon  to  infer  the  candidate 
invariants  and  using  SpaceEx  reachability  analysis  to  generate  the  reach  sets. 

For  a  given  formal  hybrid  automaton  PL  of  a  CPS,  following 
definitions  are  introduced  to  extract  the  actual  invariants  from 
candidate  invariants,  as  shown  in  Fig.  5: 

Definition  3.1:  For  a  hybrid  automaton  PL,  all  states  encoun¬ 
tered  during  executions  are  called  the  reachable  states  of  PL. 
A  state  is  already  defined  in  Definition  2.1.  Since  the  exact 
set  of  all  reachable  states  is  undecidable,  reachability  analysis 
tools  compute  the  overapproximated  sets  of  reachable  states 
(called  the  reach  sets  for  simplicity).  In  this  work,  SpaceEx 
[33]  is  used  to  compute  the  reach  sets  for  a  formal  hybrid 
automaton  PL,  denoted  by  Pin- 

Definition  3.2:  The  property  p  of  a  hybrid  automaton  PL  is 
defined  as  a  Boolean-valued  expression,  that  contains  some  or 
all  state  variables  of  PL,  and  evaluates  to  True  or  False. 

Definition  3.3:  For  a  hybrid  automaton  PL,  a  state  s  is  said 
to  satisfy  the  property  p  (i.e.,  s  t=  p)  if  p  evaluates  to  True 
when  all  state  variables  are  assigned  values  as  defined  by  the 
state  s. 

Definition  3.4:  For  a  hybrid  automaton  PL,  a  property  p  is 
an  invariant  of  PL  if  all  its  reach  sets  satisfy  p,  i.e.,  t=  p  . 
A  candidate  invariant  ip  £  $  is  also  a  property  of  PL.  Therefore, 
Definition  3.4  infers  the  actual  invariants  p  £  <b. 

Definition  3.5:  A  candidate  invariant  p  £  $  of  a  hybrid 
automaton  PL  is  the  actual  invariant  <fi  £  $  iff  1 Z-u  t=  p  £  $. 

The  candidate  invariants  for  DC  microgrids  are  obtained 
from  Hynger  in  forms  of  bounds  over  the  continuous  state 
variables,  and  denoted  as  [Bi,  Bu\.  It  is  assumed  that  SLSF 
model  depicts  the  hybrid  automaton  so  that  Hynger  can  find 
the  set  of  candidate  invariants  $.  Each  p  £  <1>  is  then 
examined  to  ascertain  whether  it  is  an  actual  invariant  as  per 
Definition  3.5,  i.e.,  checking  whether  IZ-h  C  (p  holds. 

The  FDIA  tends  to  disturb  the  consensus  and  hence  the 
invariants  as  shown  in  case  studies  in  Section  IV.  This  change 
is  employed  to  detect  FDIA  on  DC  microgrids. 

Definition  3.6:  A  hybrid  automaton  PL  is  said  to  be  operating 
under  FDIA  scenario  iff  p  <1>. 

IV.  Case  Studies 

A  small-scale  DC  microgrid  prototype  is  shown  in  Fig.  6, 
with  the  system  parameters  given  in  the  Appendix.  A  compar¬ 
ison  of  the  SLSF  model  simulation  and  the  experimental  data 
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DC  Microgrid  System  Consisting  of 
Two  DC-DC  Converters 


Computer  System  Installed 
with  dSpace  ControlDesk 


CPI  103 

Connector 

Panel 


Loads 


Fig.  6.  Experimental  setup  for  a  DC  microgrid  consisting  of  two  dc-dc 
converters  and  a  dSpace  DS1103  controller  system. 


Time,  Sec 


Fig.  7.  SLSF  simulation  and  experimental  results  for  DC  microgrid  under  no 
FDIA  scenario  showing  stable  output  current  and  output  voltage. 


is  shown  in  Fig.  7,  for  a  stable  output  under  no  FDIA  scenario. 
The  effects  of  constrained  FDIA  on  global  variables,  e.g.,  vf5*, 
and  are  shown  in  Fig.  8  and  Fig.  9,  respectively.  The 
intruder  may  also  disturb  the  consensus  protocol  when  the 
current  and  voltage  sensors  are  targeted  as  shown  in  Fig.  10 
and  Fig.  11,  respectively.  Unconstrained  FDIA  that  involves 
targeting  the  entire  set  of  global  variables,  is  very  effective  in 
destabilizing  the  DC  microgrid,  as  shown  in  Fig.  12. 

A.  FDIA  Detection 

For  FDIA  detection,  the  SLSF  model  formed  using  the 
methodology  in  Section  II  is  instrumented  using  Hynger,  and 
then  simulated  within  the  SLSF  environment  to  generate  traces 
under  no  FDIA  scenario.  This  process  generates  the  trace 


Time,  Sec 


Fig.  9.  Experimental  data  for  the  constrained  FDIA,  targeting  it?2St 


0  2  4  6  8  10  12  14  16  18 

Time,  Sec 


Fig.  10.  Experimental  data  for  the  constrained  FDIA  targeting  current  sensor. 

files,  in  Daikon  compatible  format,  that  are  passed  on  to 
Daikon.  Hence,  the  corresponding  invariants  are  generated 
automatically,  and  shown  in  Table  I.  The  SpaceEx  reachability 
analysis  tool  computes  the  reach  sets  in  the  steady  state, 
as  seen  in  Fig.  13.  It  is  shown  that  the  experimental  data 
and  the  simulation  traces  are  contained  within  the  reach 
sets.  Moreover,  reach  sets  satisfy  the  candidate  invariants 
generated  using  Hynger  under  no  FDIA  scenario.  Therefore, 
the  invariants  without  FDIA  of  Table  I  are  found  to  be  the 
actual  invariants  as  per  Definition  3.5. 

Next,  FDIA  detection  approach  is  tested  when  the  adversary 
breaks  into  the  communication  link  from  agent  2  to  agent  1 .  A 
false  data  signal  is  spoofed  into  at  time  t  =  0.6  s,  through 
the  compromised  communication  link,  x?ju  is  the  per-unit 
current  information  of  agent  2  that  is  communicated  to  agent  1 . 
The  DC  microgrid  under  FDIA  scenario  is  again  instrumented 
using  Hynger,  and  simulated  in  the  SLSF  environment  to 
generate  traces  and  the  corresponding  invariants.  The  output 
of  the  instrumented  model  under  FDIA  is  plotted  in  Fig.  14 
for  both  agents  1  and  2.  It  can  be  observed  that  the  consensus 
protocol  is  disturbed  under  FDIA. 

The  corresponding  invariants  for  the  DC  microgrid  under 


Hi  I  ■  ■  ■  ,‘^=i - ! - 1 - 1 - ! - 1 -  3 


Fig.  8.  Experimental  data  for  RELEASE;  tiiSmrBUW&N1  (UNti DflFED^1  FDIA  targeting  voltage  sensor. 
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TABLE  I 

Invariants  Without  and  With  FDIA 


Variable 

Without  FDIA 

With  FDIA 

voui 

[47.874,48.0818] 

[47.9917,48.0486] 

v°2ut 

[47.8739,48.0818] 

[47.9917,48.5258] 

■out 

ll 

[1.5071,1.7187] 

[1.418, 1.6016] 

■out 

b2 

[1.5071,1.7187] 

[1.5997,1.6175] 

Fig.  12.  Experimental  data  plots  for  the  DC  microgrid,  under  unconstrained 
FDIA,  targeting  the  entire  set  of  global  variables. 
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Fig.  13.  Phase-portrait  comparison  of  Hynger  generated  invariants,  SpaceEx, 
SLSF,  and  experimental  results,  in  the  steady  state,  for  DC  microgrid  under 
normal  conditions  (i.e.,  without  FDIA).  The  experimental  and  simulation 
results  are  contained  within  the  reach  sets  computed  using  SpaceEx.  Moreover, 
it  is  also  shown  that  the  SpaceEx  reach  sets  satisfy  the  invariants. 


FDIA  scenario  are  generated  automatically  using  Hynger.  This 
invariant  set  is  then  compared  with  the  actual  invariants,  i.e., 
invariants  under  no  FDIA  scenario  to  detect  intrusion.  A 
comparison  of  the  invariants  with  and  without  FDIA  scenario 
is  tabulated  in  Table  I.  It  is  evident  by  comparison  that  FDIA 
detection  condition  mentioned  in  Definition  3.6,  i.e.,  <p  £  <E>, 
is  satisfied  for  the  two  scenarios,  detecting  the  FDIA. 

B.  FDIA  Mitigation  Strategies 

Once  an  FDIA  is  detected,  various  mitigation  strategies 
can  suppress  the  effects  of  the  attack.  As  an  example,  three 
possible  mitigation  strategies  are  experimentally  demonstrated. 

1 )  Physical  mitigation  strategy:  The  affected  converter  may 
be  taken  offline  after  an  FDIA  is  detected.  Once  the  affected 
converter  2  is  disconnected,  proper  microgrid  operation  is 
restored,  as  shown  in  Fig.  15. 
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Fig.  15.  Experimental  data  for  the  constrained  FDIA  targeting  the  current 
sensor  of  converter  2.  The  affected  converter  is  disconnected  to  stabilize  the 
DC  microgrid. 


2 )  Communication-based  mitigation  strategy :  The  commu- 
nication  link  of  the  effected  agent  (converter)  can  be  discon¬ 
nected  so  that  other  agents  may  not  be  effected.  Once  the 
communication  link  between  the  affected  converter  2  and  non- 
affected  converter  1  is  disconnected,  the  output  of  converter  1 
is  stabilized,  as  shown  in  Fig.  16.  The  output  of  converter  2 
still  remains  unstable. 

3)  Control-based  mitigation  strategy:  One  can  use  a  modi¬ 
fied  control  scheme  to  reduce  the  effects  of  FDIA,  by  augment¬ 
ing  the  controller  with  a  false  data  suppressing  mechanism 
(e.g.,  filters  [39]).  As  shown  in  Fig.  17,  FDIA  is  initiated  at 
about  8.5  s,  and  the  modified  control  scheme  is  put  into  action 
at  about  11.97  s  to  suppress  FDIA  effects,  and  the  output  of 
the  entire  DC  microgrid  is  stabilized. 

C.  Stealthy  Attacks  with  Minimal  Weights 

The  intruder  could  potentially  fabricate  an  attack  vector 
to  bypass  the  proposed  FDIA  detection  framework,  if  the 
changes  in  candidate  invariants,  and  microgrid  operation,  are 
negligible.  This  is  demonstrated  through  the  following  two 
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Fig.  14.  The  SLSF  model  of  DC  microgrid  is  instrumented  using  Hynger, 

and  the  simulation  output  results  for  the  instrumented  model  under  FDIA  Fig.  16.  Experimental  data  for  the  constrained  FDIA  targeting  the  current 
scenario  are  shown  demonstrating  that  the  consensus  protocol  is  disturbed.  sensor  of  converter  2.  The  communication  link  between  the  affected  converter 
These  instrumented  traces  are  pagpptjpj^^  UTi’ClII  to  sta^1 ' ;/e  converter  1- 
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Fig.  17.  Experimental  data  for  the  constrained  FDIA  targeting  the  current 
sensor  of  converter  2.  As  FDIA  is  detected,  the  control  strategy  is  augmented 
with  a  false  data  suppression  mechanism.  It  is  shown  that  this  controller-based 
mitigation  action  has  stabilized  the  entire  DC  microgrid  output. 
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V.  Conclusion 

FDIA  disturbs  the  consensus  protocols  used  in  the  dis¬ 
tributed  control  of  cyber-physical  DC  microgrids.  An  FDIA 
detection  framework  is  presented  whereby  the  attack  detection 
problem  is  formalized  as  identifying  a  change  in  the  set  of 
candidate  invariants.  The  candidate  invariants  are  generated 
using  Hynger,  that  provides  an  interface  between  SLSF  models 
and  the  Daikon  tool,  which  is  an  invariant  inference  tool. 
The  hybrid  automaton  of  cyber-physical  DC  microgrid  is 
presented  to  obtain  the  reach  sets  through  reachability  anal¬ 
ysis.  Moreover,  the  SLSF  model  of  a  DC  microgrid  is  also 
developed  to  generate  the  candidate  invariants.  The  actual 
invariants  are  obtained  after  verifying  whether  the  reach  sets 
are  contained  within  the  candidate  invariants.  The  candidate 
invariants  generated  by  Hynger  are  compared  with  the  actual 
invariants  to  successfully  detect  FDIA. 


Appendix 


Buck  converter  parameters  are  L  =  2.64  mil ,  C  =  2.2  mF , 
and  Fs  =  60  kHz.  The  local  loads  are  R\  =  IF  =  30  Q.  The 
transfer  functions  are  given  by: 


Ti 


1 

0.01s +  1’ 


tt2 


1 

0.05s +  1' 


(25) 


Fig.  18.  FDIA,  targeting  the  current  sensor  of  converter  2,  with  the  minimal- 
weight  attack  vector  that  can  be  detected  using  this  framework. 


experiments.  First,  an  attack  vector  with  small  weights  is  de¬ 
signed  that  can  be  detected  using  the  proposed  framework.  The 
invariants  for  the  output  current  generated  using  Hynger  are 
■oui  _  [1.55,1.77]  anc[  i°ut  =  [1.55,1.77],  These  invariants 
are  deviated  from  the  corresponding  actual  invarinats  tabulated 
in  Table  I,  indicating  the  presence  of  an  FDIA.  The  negative 
effects  of  this  FDIA  are  shown  in  Fig.  18.  It  is  demonstrated 
that  an  FDIA  with  such  minimal  destabilizing  effects  can  still 
be  detected  using  the  proposed  framework.  Next,  an  attack 
vector  with  smaller  weights  is  fabricated  to  bypass  through 
this  FDIA  framework.  The  invariants  for  the  output  current 


generated  using  Hynger  are 
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The  DC  microgrid  parameters  are:  Vref  =  [48  48] T,  Imax  = 


[4 

25 


0.05 


C, 

0  1 
Ll  0 
1 


Vin  =  [80 


,  Prn.r  0.01,  Im.r.  L 


B  =  A,C  =  0.5A,  /  =  3 


1 

L0 


A  = 
P  = 


=  [1.5071,1.7187]  and 


i°ut  =  [1.5071, 1.7188]  that  are  comparable  with  the  actual 
invariants  in  Table  I,  hence  missing  the  FDIA.  However,  the 
negative  effects  of  this  FDIA  are  negligible,  as  seen  in  Fig.  19, 
as  they  do  not  disturb  the  microgrid  operation. 
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Abstract.  Designing  complex  systems  using  graphical  models  in  so¬ 
phisticated  development  environments  is  becoming  de-facto  engineering 
practice  in  the  cyber-physical  system  (CPS)  domain.  Development  envi¬ 
ronments  thrive  to  eliminate  bugs  or  undefined  behaviors  in  themselves. 
Formal  techniques,  while  promising,  do  not  yet  scale  to  verifying  entire 
industrial  CPS  tool  chains.  A  practical  alternative,  automated  random 
testing,  has  recently  found  bugs  in  CPS  tool  chain  components.  In  this 
work  we  identify  problematic  components  in  the  Simulink  modeling  en¬ 
vironment,  by  studying  publicly  available  bug  reports.  Our  main  contri¬ 
bution  is  CyFuzz,  the  first  differential  testing  framework  to  find  bugs  in 
arbitrary  CPS  development  environments.  Our  automated  model  gener¬ 
ator  does  not  require  a  formal  specification  of  the  modeling  language. 
We  present  prototype  implementation  for  testing  Simulink,  which  found 
interesting  issues  and  reproduced  one  bug  which  MathWorks  fixed  in  sub¬ 
sequent  product  releases.  We  are  working  on  implementing  a  full-fledged 
generator  with  sophisticated  model-creation  capabilities. 

Keywords:  Differential  testing,  cyber-physical  systems,  model-based 
design,  Simulink 


1  Introduction 

Widely  used  cyber-physical  system  (CPS)  development  tool  chains  are  complex 
software  systems  that  typically  consist  of  millions  of  lines  of  code  [1] .  For  exam¬ 
ple,  the  popular  MathWorks  Simulink  tool  chain  contains  model-based  design 
tools  (in  which  models  in  various  expressive  modeling  languages  are  used  to 
describe  the  overall  system  under  control  [2]),  simulators,  compilers,  and  auto¬ 
mated  code  generators.  Like  any  complex  piece  of  code,  CPS  tool  chains  may 
contain  bugs  and  such  bugs  may  lead  to  severe  CPS  defects. 

The  vast  majority  of  resources  in  the  CPS  design  and  development  phases  are 
devoted  to  ensure  that  systems  meet  their  specifications  [3,4].  In  spite  of  hav¬ 
ing  sophisticated  design  validation  and  verification  approaches  (model  checking, 
automated  test  case  generation,  lrardware-in-the-loop  and  software-in-tlie-loop 
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testing  etc.),  we  see  frequent  safety  recalls  of  products  and  systems  among  in¬ 
dustries,  due  to  CPS  bugs  [5-7]. 

Since  many  CPSs  operate  in  safety-critical  environments  and  have  strict  cor¬ 
rectness  and  reliability  requirements  [8],  it  would  be  ideal  for  CPS  development 
tools  to  not  have  bugs  or  unintended  behaviors.  However,  this  is  not  generally 
true  as  demonstrated  by  recent  random  testing  projects  finding  bugs  in  a  static 
analysis  tool  (Frama-C)  [9]  and  in  popular  C  compilers  (GCC  and  LLVM)  [10], 
which  are  widely  used  in  CPS  model-based  design. 

It  would  be  extremely  expensive  or  possibly  even  practically  infeasible  to 
formally  verify  entire  CPS  tool  chains.  In  addition  to  their  sheer  size  in  terms  of 
lines  of  code,  a  maybe  more  significant  hurdle  is  the  lack  of  a  complete  and  up 
to  date  formal  specification  of  the  CPS  tool  chain  semantics,  which  may  be  due 
to  their  complexity  and  rapid  release  cycles  [1, 11]. 

Instead  of  formally  verifying  the  absence  of  bugs  in  all  CPS  tool  chain  exe¬ 
cution  paths,  we  revert  to  showing  the  presence  of  bugs  on  individual  paths  (aka 
testing),  which  can  still  be  a  major  contributor  to  software  quality  [12].  Differen¬ 
tial  testing  or  fuzzing ,  a  form  of  random  testing,  mechanically  generates  random 
test  inputs  and  presents  them  to  comparable  variations  of  a  software  [12].  The 
results  are  then  compared  and  any  variation  from  the  majority  (if  one  exists) 
likely  indicates  a  bug  [13].  This  scheme  has  been  effective  at  finding  bugs  in 
compilers  and  interpreters  of  traditional  programming  languages.  As  an  exam¬ 
ple,  various  fuzzing  schemes  have  collectively  found  over  1,000  bugs  in  widely 
used  compilation  tools  such  as  GCC  [10, 11, 14]. 

While  compiler  testing  is  promising,  when  testing  CPS  tool  chains  we  face 
additional  challenges  beyond  what  is  covered  by  testing  compilers  of  traditional 
programming  languages  (such  as  Csrnith  creating  C  programs),  since  CPS  mod¬ 
eling  languages  differ  significantly  from  traditional  programming  languages.  A 
key  difference  is  that  the  complete  semantics  of  widely  used  commercial  modeling 
languages  (e.g.,  MathWorks  Simulink  and  Stateflow  [15])  are  not  publicly  avail¬ 
able  [1, 16, 17].  Moreover,  modeling  language  semantics  often  depend  on  subtle 
details,  such  as  two-dimensional  layout  information,  internal  model  component 
settings,  and  the  particular  interpretation  algorithm  of  simulators  [1].  Finally, 
random  generation  of  test  cases  for  CPS  development  environments  has  to  ad¬ 
dress  a  combination  of  programming  paradigms  (e.g.,  both  graphical,  data-flow 
language  and  textual  imperative  programming  language  in  the  same  model), 
which  is  rare  in  traditional  compiler  testing. 

Since  existing  testing  and  verification  techniques  are  not  sufficient  for  ensur¬ 
ing  the  reliability  of  CPS  tool  chains,  we  propose  CyFuzz:  a  novel  conceptual 
differential  testing  framework  for  testing  arbitrary  CPS  development  environ¬ 
ments.  We  use  the  term  system  under  test  (SUT)  to  refer  to  the  CPS  tool  chain 
being  tested.  CyFuzz  has  a  random  model  generator  which  automatically  gener¬ 
ates  random  CPS  models  the  SUT  may  simulate  or  compile  to  embedded  native 
code.  CyFuzz’s  comparison  framework  component  then  detects  dissimilarity  (if 
it  exists)  in  the  results  obtained  by  executing  (or,  simulating)  the  generated 
model,  by  varying  components  of  the  SUT. 
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We  also  present  an  implementation  for  testing  the  Simulink  environment, 
which  is  widely  used  in  CPS  industries  for  model-based  design  of  dynamic  and 
embedded  systems  [18, 19].  Although  our  current  prototype  implementation  tar¬ 
gets  Simulink,  the  described  conceptual  framework  is  not  tool  specific  and  should 
thus  be  applicable  to  related  CPS  tool  chains,  such  as  NFs  Lab  VIEW  [20]. 

To  the  best  of  our  knowledge,  CyFuzz  is  the  first  differential  testing  frame¬ 
work  for  fuzzing  CPS  tool  chains.  To  address  the  problem  of  missing  formal 
semantics  during  model  generation,  we  follow  a  simple,  feedback-driven  model 
generation  approach  that  iteratively  fixes  generated  models  according  to  the 
SUT’s  error  descriptions.  To  summarize,  this  paper  makes  the  following  contri¬ 
butions: 

—  To  understand  the  types  of  Simulink  bugs  that  affect  users,  we  first  analyze 
a  subset  of  the  publicly  available  Simulink  bug  reports  (Section  3). 

—  We  present  CyFuzz,  a  conceptual  framework  for  (1)  generating  random  but 
valid  models  for  a  CPS  modeling  language,  (2)  simulating  the  generated 
models  on  alternative  CPS  tool  chain  configurations,  and  (3)  comparing  the 
simulation  results  (Section  4).  We  then  describe  interesting  implementation 
details  and  challenges  of  our  prototype  implementation  for  Simulink  (Sec¬ 
tion  5). 

—  We  report  on  our  experience  of  running  our  prototype  tool  on  various  Simulink 
configurations  (Section  6),  identifying  comparison  errors  and  semi-independently 
reproducing  a  confirmed  bug  in  Simulink’s  Rapid  Accelerator  mode. 


2  Background:  Model-based  CPS  Design  and  Simulink 

This  section  provides  necessary  background  information  on  model-based  devel¬ 
opment.  We  define  the  terms  used  for  explaining  a  conceptual  differential  testing 
framework  and  subsequently  relate  them  with  Simulink. 


2.1  CPS  Model  Elements 

The  following  concepts  and  terms  are  applicable  to  many  CPS  modeling  lan¬ 
guages  (including  Simulink).  A  model ,  also  known  as  a  block-diagram,  is  a  math¬ 
ematical  representation  of  some  CPS  [18].  Designing  a  diagram  starts  with  choos¬ 
ing  elementary  elements  called  blocks.  Each  block  represents  a  component  of  the 
CPS  and  may  have  input  and  output  ports.  An  input  port  accepts  data  on  which 
the  block  performs  some  operation.  An  output  port  passes  data  to  other  input 
ports  using  connections.  An  output  port  can  be  connected  to  more  than  one  in¬ 
put  port  while  the  opposite  is  not  true  in  general.  A  Block  may  have  parameters, 
which  are  configurable  values  that  influence  the  block’s  behavior.  Somewhat  sim¬ 
ilar  to  a  programming  language’s  standard  libraries,  a  CPS  tool  chain  typically 
provides  block  libraries,  where  each  library  consists  of  a  set  of  predefined  blocks. 

Since  hierarchical  models  are  commonly  found  in  industry,  CyFuzz  supports 
generating  such  models  as  well.  This  can  be  achieved  by  grouping  some  blocks 
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of  a  model  together  and  replacing  them  by  a  new  block  which  We  call  a  child , 
whereas  the  original  model  is  called  parent. 

When  simulating,  the  SUT  numerically  solves  the  mathematical  formulas 
represented  by  the  model  [18].  Simulation  is  usually  time  bound  and  at  each 
step  of  the  simulation,  a  solver  calculates  the  blocks’  outputs.  We  use  the  term 
signal  to  mean  output  of  a  block’s  port  at  a  particular  simulation  step. 

The  very  first  phase  of  the  simulation  process  is  compiling  the  model.  This 
stage  also  looks  for  incorrectly  generated  models  and  raises  failures  for  syntactical 
model  errors,  such  as  data  type  mismatches  between  connected  output  and  input 
ports.  If  an  error  is  found  in  the  compilation  phase,  the  SUT  does  not  attempt 
simulating  the  model.  After  successful  simulation,  code  generators  can  generate 
native  code,  which  may  be  deployed  in  target  hardware  [1]. 

2.2  Example  CPS  Development  Environment:  Simulink 

While  our  conceptual  framework  uses  the  above  terms,  they  also  apply  directly 
in  the  context  of  Simulink  [21].  Besides  having  a  wide  selection  of  built-in  blocks, 
Simulink  allows  integrating  native  code  (e.g.,  Matlab  or  C  code)  in  a  model  via 
Simulink’s  S-function  interface,  which  lets  users  create  custom  blocks  for  use 
in  their  models.  Simulink’s  Subsystem  and  Model  referencing  features  enable 
hierarchical  models. 

Simulink  has  three  simulation  modes.  In  Normal  mode,  Simulink  does  not 
generate  code  for  blocks,  whereas  it  generates  native  code  for  certain  blocks  in 
the  Accelerator  mode.  Unlike  in  these  two  modes,  the  Rapid  Accelerator 
mode  further  creates  for  the  model  a  standalone  executable.  To  capture  sim¬ 
ulation  results  we  use  Simulink’s  Signal  Logging  functionality  as  we  found 
implementing  it  quite  feasible.  However,  for  cases  where  the  approach  is  not 
applicable  (see  [21]),  we  use  Simulink’s  sim  api  to  record  simulation  data. 

3  Study  of  Existing  Bugs:  Incorrect  Code  Generation 

To  understand  the  types  of  bugs  Simulink  users  have  found  and  care  about,  we 
performed  a  study  on  the  publicly  available  bug  reports  from  the  MathWorks 
website1.  We  identified  commonalities  in  bug  reports,  which  we  call  classifica¬ 
tion  factors.  We  limited  our  study  to  bug  reports  found  via  the  search  query 
incorrect  code  generation ,  as  earlier  studies  have  identified  code  generation  as 
vulnerable  [1,22]. 

We  investigated  bug  reports  affecting  Matlab/Simulink  version  2015a  as  we 
were  using  it  in  our  experiments.  As  of  February  17,  2016,  there  were  50  such  bug 
reports,  among  which  47  have  been  fixed  in  subsequent  releases  of  the  products. 
Table  1  summarizes  the  findings.  Our  complete  study  data  are  available  at: 

http : / /bit . ly/simstudy 

Table  1  shows  only  those  classification  factors  that  affect  at  least  20%  of 
all  the  bug  reports  that  we  have  studied.  We  use  insights  obtained  from  the 

1  Available:  http://www.mathworks.com/support/bugreports/ 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

122 


A  Differential  Testing  Framework  for  CPS  Development  Environments  5 

Table  1.  Study  of  publicly  available  Simulink  bug  reports.  The  right  column  denotes 
the  percentage  of  bug  reports  affected  by  a  the  given  classification  factor.  Each  bug 
report  may  be  classified  under  multiple  factors. 


Classification  factor  Bugs  [%] 

Reproducing  the  bug  requires  a  code  generator  to  generate  code  60 

Reproducing  the  bug  requires  specific  block  parameter  values  and/or 
port  or  function  argument  values  and  data-types 

Reproducing  the  bug  requires  comparing  simulation-result  and  gener- 
ated  code’s  output 

Reproducing  the  bug  requires  connecting  the  blocks  in  a  particular  way  36 

Reproducing  the  bug  requires  specific  model  configuration  settings  32 

Reproducing  the  bug  requires  hierarchical  models  24 

Reproducing  the  bug  requires  built-in  Matlab  functions  20 


study  in  our  CyFuzz  prototype  implementation.  For  example,  many  of  the  bug 
reports  (54%)  are  related  to  simulation  result  and  generated  code  execution  out¬ 
put  mismatch.  Thus,  differential  testing  (e.g.,  by  comparing  simulation  and  code 
execution)  seems  like  a  good  fit  for  finding  bugs  in  CPS  tool  chains.  Further  in¬ 
sight  that  is  reflected  in  our  tool  is  that  it  is  worth  exploring  the  large  space 
of  possible  block  connections  (36%  of  bug  reports)  e.g.,  via  random  block  and 
connection  generation.  Other  insights  we  want  to  use  in  the  future  are  to  incor¬ 
porate  random  block  parameter  values  and  port  data-types  (56%)  and  model 
configurations  (32%). 


4  Differential  Testing  of  CPS  Development  Tool  Chains 


(Model)  (Model)  (Model)  (Data) 
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Blocks 


Pn\ 

\  mj 

Jfj 

□ 

_ r 

* 

Connect 

Ports 

* 

Fix  Errors 
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Random  Model  Generator  Comparison  Framework 


* 


Fig.  1.  Overview  of  the  differential  testing  framework.  The  first  three  phases  cor¬ 
respond  to  the  random  model  generator,  while  the  rest  belongs  to  the  comparison 
framework. 


At  a  high  level  we  can  break  our  objective  into  two  sub  goals:  creating  a 
random  model  generator  and  defining  a  comparison  framework.  We  first  present  a 
theory  applicable  to  a  conceptual  CPS  framework  in  this  section.  Fig.  1  provides 
a  schematic  overview  of  CyFuzz’s  processing  phases.  The  first  three  phases  belong 
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to  the  random  model  generator,  and  the  remaining  two  constitute  the  comparison 
framework.  The  first  two  phases  create  a  random  model  (which  may  violate 
Simulink’s  model  construction  rules) .  The  third  phase  fixes  many  of  these  errors, 
such  that  the  model  passes  the  SUT’s  type  checkers  and  the  SUT  can  simulate 
it.  If  it  succeeds  it  passes  the  model  to  the  fourth  phase  to  simulate  the  model 
in  various  SUT  configurations  and  to  record  results.  The  final  phase  detects  any 
dissimilarities  in  the  collected  data,  which  we  call  comparison  error  bugs. 

4.1  Conceptual  Random  Model  Generator 

Following  are  details  on  the  generator’s  three  phases. 

Listing  1.1.  Select  Blocks  phase  of  the  conceptual  random  model  generator. 

method  select_blocks  (n,  block -libraries): 

/*  Choose  n  blocks  from  the  given  block-libraries,  place  the  blocks 
in  a  new  model,  configure  the  blocks,  and  return  the  model.  */ 
in  =  create_emptyjmodel()  //  New,  empty  model 
blocks  =  choose_blocks(n,  block-libraries)  II  N  from  block-libraries 
for  each  block  b  in  blocks: 

place_blockdnjmodel(m,  b) 
configure_block(6,  n,  block-libraries) 
return  m 


Select  Blocks.  Listing  1.1  summarizes  this  phase,  which  selects,  places,  and 
configures  the  model’s  blocks.  The  generator  has  a  list  of  block  libraries  and 
for  each  library  a  predetermined  weight.  Using  the  weights,  the  choosejblocks 
method  selects  n  random  blocks.  The  value  n  can  be  fixed  or  randomly  selected 
from  a  range.  On  a  newly  created  model  the  generator  next  places  each  of  these 
blocks  using  the  place-block-in-model  method.  For  creating  inputs,  CyFuzz  se¬ 
lects  various  kinds  of  blocks,  to,  for  example,  provide  random  inputs  to  the 
model. 

The  con  figure -block  method  selects  block  parameter  values  and  satisfies 
some  block  constraints  (e.g.,  by  choosing  blocks  required  for  placing  a  certain 
block).  For  creating  hierarchical  models,  a  child  model  is  considered  as  a  regular 
block  in  the  parent  model  and  is  passed  as  a  parameter  to  configure -block, 
which  calls  select-blocks  to  create  a  new  child  model.  Here  n  is  equal  to  the 
parent  model,  but  block-libraries  may  not  be  the  same  (e.g.,  certain  blocks  are 
not  allowed  in  some  Simulink  child  models). 


Connect  Ports.  The  second  phase  follows  a  simple  approach  to  maximize  the 
number  of  ports  connected.  CyFuzz  arbitrarily  chooses  an  output  and  an  input 
port  from  the  model’s  blocks,  prioritizing  unconnected  ports.  It  then  connects 
them  and  continues  the  process  until  all  input  ports  are  connected.  Consequently, 
some  output  ports  may  be  left  unconnected. 
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Listing  1.2.  fix-errors  tries  to  fix  the  model  errors  that  the  simulate  method  raises; 
p  is  a  SUT  configuration;  t  denotes  a  timeout  value. 

method  fix_errors  (m,  p,  attempt -limit,  t ): 
for  i  =  1  to  attempt -limit: 

<  Status’ r data’ errors  >  =  simulate(ro,  p,  t) 
if  ^status  is  error: 

if  fixjmodel(m,  errors')  is  false: 
return  <  rptatus,  rpdata,  errors  > 

else: 

return  <  rpstatus,rpdata,  errors  > 

return  simulate(m,  p,  t ) 


Fix  Errors.  Because  of  their  simplicity,  CyFuzz’s  first  two  phases  may  generate 
invalid  models  that  cannot  be  simulated  successfully.  The  third  phase  tries  to 
fix  these  errors.  Listing  1.2  outlines  the  approach.  It  uses  method  simulate  to 
simulate  model  m  up  to  time  t  G  IR+  (in  milliseconds)  using  SUT  configuration  p. 

The  simulate  output  is  a  3-tuple,  where  rptatus  is  one  of  success,  error, 
or  timed  —  out.  Note  that  first  step  of  simulation  is  compiling  the  model  (see 
Section  2).  If  m  has  errors,  simulate  will  abort  compilation,  storing  error-related 
diagnostic  information  in  errors,  r d  .  contains  simulation  results  (time  series 
data  of  the  model’s  blocks’  outputs)  if  rptatus  =  success. 

At  this  point  we  assume  that  the  error  messages  are  informative  enough 
to  drive  the  generator.  For  example,  Simulink  satisfies  this  assumption.  Using 
errors,  fix-model  tries  to  fix  the  errors  by  changing  the  model.  As  it  changes 
the  model  this  phase  may  introduce  new  errors.  We  try  to  address  such  sec¬ 
ondary  errors  in  subsequent  loop  iterations  in  Listing  1.2,  up  to  a  configurable 
number  attempt-limit.  While  this  approach  is  clearly  an  imperfect  heuristic,  it 
has  worked  relatively  well  in  our  preliminary  experience  (as,  e.g.,  is  indicated  by 
the  low  error  rate  in  Table  2). 

4.2  Conceptual  Comparison  Framework 

Here  we  explore  simulating  a  randomly  generated  model  varying  SUT-specific 
configuration  options  of  a  CPS  tool  chain,  and  thus  testing  it  in  two  phases. 


Log  Signals.  If  simulation  was  successful  in  the  Fix  Errors  phase,  CyFuzz  sim¬ 
ulates  the  model  varying  configurations  of  the  SUT  in  this  phase;  let  P  be  such 
a  set  of  configurations.  Using  the  simulate  method  introduced  in  Section  4.1,  for 
each  p  G  P  we  calculate  <  rptatus,  rdata,  errors  >=  simulate(m,p ,  t)  for  a  model 
to  and  add  rdata  to  a  set  d  only  if  rptatus  =  success.  We  pass  d  to  next  phase 
of  the  framework.  rdt  should  contain  time  series  data  of  the  output  ports  of 
the  model’s  blocks  at  all  available  simulation  steps.  In  the  next  phase,  however, 
we  use  only  the  values  recorded  at  the  last  simulation  step;  we  leave  comparing 
signal  values  at  other  simulation  steps  as  future  task. 
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Compare.  In  its  last  phase,  CyFuzz  compares  the  recorded  simulation  results 
d  obtained  in  the  previous  phase  using  method  compare  (Listing  1.4).  It  uses 
method  retrieve,  which  returns  the  signal  value  of  a  particular  block’s  particular 
port  at  a  given  time  instance.  If  the  value  is  not  available  (e.g.,  blocks  that  do 
not  have  output  ports  do  not  participate  in  signal  logging),  it  returns  the  special 
value  Nil.  compare  also  uses  method  latest-time  which  returns  the  time  of  the 
last  simulation  step  for  a  given  block’s  particular  port.  If  no  data  is  available,  it 
returns  Nil. 


Listing  1.3.  Determining  equivalence  via  tolerance  limit  t. 
method  equiv  ( p ,  q ): 

if  p  and  q  are  Nil:  //  Missing  both  data  points 
return  true 

if  p  or  q  is  Nil:  //  Missing  one  data  point 
return  false 
return  \p  —  q\  <  e 


Listing  1.4.  This  method  compares  two  execution  results  (of  model  m)  taken  as  first 
two  arguments  and  throws  errors  if  it  finds  a  dissimilarity. 

method  compare  (rPdata,  rqdata,  to): 
for  each  block  b  of  the  model  to: 

for  each  output  port  y  of  the  block  b: 
t.p  =  latest_time(r^ata,  b,  y ) 
tq  =  latest _time(r«ota>  b,  y) 
if  equiv(tp,tq)  is  false: 

throw  “Time  Mismatch”  error 
else  if  tp  ^  Nil: 

if  equiv  (retrieve(rdata,  b,  y,  tp),  retrieve(rdata,  b,  y,  tq))  is  false 

throw  “Data  Mismatch”  error 

Now,  taking  two  elements  from  d  at  a  time  we  form  all  possible  pairs  (rdata,  rdata ) 
where  p  ^  q  and  apply  method  compare  on  them.  As  comparing  floating-point 
numbers  using  straight  equality  checking  is  problematic  [1,  23],  eqiv  (Listing  1.3) 
method  uses  a  tolerance  limit  to  determine  floating-point  equivalence.  If  compare 
reports  an  error,  we  mark  to  as  a  comparison  error  for  p,  q  and  submit  it  to 
manual  inspection. 


5  CyFuzz  Prototype  Implementation  for  Simulink 

We  have  developed  a  prototype  implementation  of  CyFuzz  mostly  in  Matlab. 
The  tool  continuously  generates  one  Simulink  model  at  a  time  and  then  passes 
it  to  the  comparison  framework.  Source  code,  implementation  and  usage  de¬ 
tails,  sample  generated  models,  and  detailed  experiment  results  are  available  at: 

https : / /github . com/verivital/ slsf _randgen. 
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Selecting  and  Configuring  Blocks.  Simulink  itself  has  over  15  built-in  libraries. 
MathWorks  also  offers  toolboxes,  which  add  to  Simulink  additional  libraries. 
To  date  we  have  included  in  our  experiments  blocks  from  only  four  of  these 
libraries,  Sources,  Sinks,  Discrete,  and  Concrete.  We  use  default  parameter 
values  for  configuring  most  blocks.  However,  some  Simulink  blocks  do  not  allow 
placing  multiple  instances  of  the  same  block  with  the  same  default  value  in  a 
model.  For  these  blocks  we  randomly  choose  parameter  values. 

Generating  Hierarchical  Models.  Since  hierarchical  models  are  very  popular 
among  Simulink  users,  our  prototype  can  generate  them.  Currently,  the  generator 
uses  Model  referencing  and  For  each  subsystems  blocks  to  create  hierarchi¬ 
cal  models.  CyFuzz  generates  model  hierarchies  up  to  a  configurable  depth.  In 
doing  so  it  places  and  configures  related  blocks.  For  example,  CyFuzz  automati¬ 
cally  puts  input  (output)  related  blocks  in  a  new  child  model  which  are  used  to 
accept  (return)  data  from  (to)  the  parent  model.  The  number  of  blocks  for  the 
top-level  and  child  models  are  chosen  randomly  from  user-provided  ranges. 

Fix  Errors  Phase.  We  utilize  Matlab’s  exception  handling  mechanism  to  learn 
what  prevented  successful  compilation  of  the  model.  Some  information  (e.g., 
the  error  type)  can  be  directly  collected  from  the  exception.  Collecting  other 
important  information,  such  as  the  actual  problematic  block,  can  be  nontrivial. 
For  example,  for  algebraic  loop  errors  sometimes  CyFuzz  has  to  identify  other 
blocks  (e.g.,  a  parent  block)  to  fix  the  problem.  As  another  example,  the  current 
CyFuzz  version  does  not  attempt  to  know  the  data  types  of  the  ports  in  the 
Connect  Ports  phase.  Rather,  it  collects  such  information  when  compiling  the 
model  using  diagnostic  information  returned  by  the  SUT. 

Models  with  Random  Native  Code.  To  facilitate  blocks  with  custom  behavior, 
Simulink  allows  placing  native  code  (C,  Matlab  etc.)  directly  in  models.  To  gen¬ 
erate  such  blocks  we  leverage  Csrnith,  which  generates  random  C  programs  [10]. 
We  designed  simple  Simulink  blocks  using  Matlab’s  S-function  interface  that 
use  random  code  generated  by  a  customized  version  of  Csmitli.  Our  customized 
version  is  capable  of  generating  many  different  C  functions  that  can  be  called 
from  various  simulation  steps.  We  looked  for  both  crash  errors  and  “wrong  code 
errors”  (similar  to  our  comparison  error).  However,  this  is  not  fully  integrated 
with  CyFuzz  yet. 

The  Comparison  Framework.  CyFuzz  starts  with  varying  simulation  modes  (see 
Section  2.2).  and  compiler  optimization  levels.  For  instance,  “Normal  mode”, 
“Accelerator  mode;  optimization  on”,  and  “Rapid  Accelerator;  optimization 
off”  are  options  to  vary.  Varying  compilers,  code  generators,  solver-specific  set¬ 
tings,  and  other  possible  SUT  configuration  options  are  future  work. 

6  Experience  with  CyFuzz 

Here  we  analyze  our  prototype  implementation  based  on  experimental  results. 
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6.1  Research  Questions  (RQ),  Experimental  Setup,  and  Results 

Throughout  this  work  we  explore  the  following  research  questions. 

RQ1  Is  the  random  model  generator  effective?  Which  portion  of  the  generated 
models  can  the  SUT  compile  and  simulate  within  a  given  time  bound? 
RQ2  Using  the  generated  models,  can  the  comparison  framework  effectively 
find  bugs  (comparison  errors  or  crashes)  in  the  SUT  ? 

RQ3  What  is  the  runtime  of  each  of  CyFuzz’s  stages?  Does  the  generator  scale 
with  the  generated  model’s  number  of  blocks? 

To  answer  these  questions  we  conducted  experiments  using  Matlab  2015a  on 
Ubuntu  14.10  and  varied  simulation  mode  (Normal  vs.  Accelerator)  and  opti¬ 
mizer  (on  vs.  off)  for  the  later  mode.  For  the  fix-errors  method  (Listing  1.2) 
we  chose  attempt  limit  10  and  timeout  12.  For  choosing  blocks  we  used  a  tra¬ 
ditional  0(n)  implementation  of  the  fitness  proportion  selection  algorithm  [24]. 
We  have  not  included  in  these  experiments  hierarchical  models  or  custom  blocks. 


Table  2.  Each  row  represents  a  separate  experiment.  Columns  3-6  is  the  percentage 
of  blocks  selected  per  library  (e.g.,  experiment  A  chose  80%  of  the  blocks  from  the 
Discrete  library).  Error  denotes  the  number  of  models  that  failed  to  simulate.  Timed- 
out  denotes  the  models  that  did  not  complete  simulation  within  the  time  bound. 


Exp. 

Total  Discrete 

Concrete 

Source 

Sink 

error 

timed-out 

Confirmed 

Label  Models 

[%] 

[%] 

[%] 

[%] 

[%] 

[%] 

Bugs  [%] 

A 

1172 

80 

0 

10 

10 

9.73 

0.60 

0 

B 

1095 

43 

37 

10 

10 

1.74 

7.03 

0 

C 

1449 

0 

80 

10 

10 

12.01 

8.63 

0 

Table  3.  More  information  on  experiments  from  Table  2.  Columns  3-7  denotes  the 
time  taken  by  the  five  phases  of  CyFuzz.  Runtime  denotes  the  average  time  CyFuzz 
spent  for  a  model. 


Exp.  Blocks/  Select  Connect  Fix  Log  Compare  Runtime 

Label  Model  Blocks  [%]  ports  [%]  Errors  [%]  Signals  [%]  [%]  [sec] 


A 

35.00 

7.85 

0.64 

16.00 

74.55 

0.96 

40.37 

B 

34.96 

6.06 

0.39 

16.06 

76.86 

0.63 

51.87 

C 

35.05 

8.09 

0.51 

11.02 

79.58 

0.80 

42.51 

Effectively  Creating  Random  Models  (RQ  1).  As  the  experimental  results  in  Ta¬ 
ble  2  suggest,  our  tool  can  generate  many  models  that  Simulink  can  successfully 
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simulate.  For  each  row  in  the  table  we  have  a  low  error  and  timed-out  rate.  This 
high  success  rate  is  crucial  for  the  framework  as  it  only  uses  such  valid  models  in 
the  tool’s  later  comparison  framework  phases.  We  also  observed  that  the  number 
of  errors  and  timed-out  models  varied  with  the  selected  block  libraries,  but  we 
have  not  yet  analyzed  the  reasons  of  these  variations. 

Effectiveness  of  Comparison  Framework  (RQ  2).  We  have  not  found  new  bugs 
yet,  however,  our  framework  reproduced  an  existing  bug  and  found  interesting 
cases  (see  Section  6.2). 

Runtime  Analysis  (RQ  3).  The  Select  Blocks  algorithm  of  Listing  1.1  has  run¬ 
time  0(n),  n  being  the  number  of  blocks  in  the  model  and  using  an  0(1)  block 
selection  algorithm.  The  random  model  generator  scales  linearly  with  the  number 
of  blocks.  But  as  the  number  of  blocks  grows,  the  number  of  timed-out  models 
and  errors  also  grow.  A  preliminary  analysis  suggests  that  there  are  relatively 
few  distinct  error  causes.  We  group  errors  by  their  causes  and  fixing  one  cause 
dramatically  increased  the  overall  number  of  successfully  executed  models. 

Table  3  indicates  that  the  Log  Signals  phase  uses  most  of  the  runtime.  This 
result  is  not  surprising,  as  in  this  phase  the  SUT  simulates  the  model,  generates 
and  executes  code,  and  logs  the  data,  all  of  which  are  time  consuming  tasks. 


Using  Native  Code/Custom  Blocks.  In  separate  experiments  we  used  a 
fixed  Simulink  model  with  a  custom  block  created  using  S-Function.  We  re¬ 
peatedly  generated  random  C  code  using  a  customized  version  of  Csrnith  and 
plugged  this  code  in  the  S-function,  which  effectively  ran  the  code  once  we  sim¬ 
ulated  the  model.  We  used  different  optimizer  settings  for  GCC  when  compiling 
and  were  able  to  reproduce  crash  and  “wrong  code”  bugs  of  GCC  4.4.3.  This 
shows  that  incorporating  Csrnith  in  our  framework  is  promising.  However,  more 
work  is  needed  to  fully  utilize  Csmith-generated  programs  and  create  sophisti¬ 
cated  Simulink  blocks  using  them.  One  limitation  is  that  floating-point  support 
in  Csrnith  is  currently  still  basic  and  can  only  be  used  for  detecting  crash-bugs. 

6.2  Interesting  Comparison  Framework  Findings 

Following  are  two  interesting  findings  of  our  experiments,  including  one  inde¬ 
pendently  rediscovered  confirmed  Simulink  bug. 


Comparison  Error  for  Models  with  Algebraic  Loops.  In  our  experiments 
we  noticed  comparison  errors  for  some  models  where  Simulink  solved  algebraic 
loops.  Investigating  further  we  noticed  that  when  Simulink  solves  an  algebraic 
loop  it  is  not  confident  of  its  correctness  [21].  For  this,  we  did  not  classify  this  case 
as  a  bug.  CyFuzz  now  eliminates  algebraic  loops  altogether  rather  than  relying 
on  Simulink  to  solve  them.  We  note  that  one  can  use  our  tool  to  opportunistically 
discover  such  inaccuracies  for  models  with  algebraic  loops  and  decide  whether 
to  accept  Simulink’s  solution  for  solving  the  loops. 
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Fig.  2.  Screen-shot  of  generated  top-level  Simulink  model  which  reproduced  a  bug 


Bug  in  Simulink’s  Rapid  Accelerator  Mode.  In  separate  experiments  with 
hierarchical  models,  we  noticed  that  for  a  model  (see  Fig.  2)  values  of  a  Simulink 
Outport  block  are  significantly  different  in  Normal  and  Rapid  Accelerator 
mode.  This  was  detected  automatically  by  our  comparison  framework.  After 
submitting  a  bug  report  MathWorks  confirmed  that  the  case  was  already  iden¬ 
tified  as  a  bug  and  they  fixed  it  for  later  versions. 

7  Future  Work  and  Discussion 

Our  ultimate  goal  is  to  provide  a  full-fledged  fuzz-testing  framework  for  Simulink. 
Our  work  on  CyFuzz  and  our  prototype  implementation  for  Simulink  are  thus 
both  ongoing.  Following  is  a  sample  of  the  opportunities  for  improvement. 

The  current  prototype  implementation  has  several  limitations.  Currently,  the 
tool  chooses  blocks  from  only  four  built-in  libraries.  Incorporating  additional  li¬ 
braries  will  increase  the  expressiveness  of  generated  models  and  thus  its  potential 
for  finding  bugs.  Also,  we  plan  on  integrating  custom  blocks  developed  using  na¬ 
tive  code  and  perform  experiments  we  were  not  able  to  conduct  yet. 

The  comparison  framework  implementation  is  also  not  free  from  shortcom¬ 
ings.  So  far,  we  have  only  used  various  simulation  modes  and  compiler  opti¬ 
mization  levels.  However,  we  are  interested  in  adding  more  variations  (e.g.  those 
listed  in  Section  5).  Finally,  CyFuzz  should  compare  signals  in  multiple  simula¬ 
tion  steps,  since  it  was  also  found  effective  in  previous  work  [25]. 

8  Related  Work 

The  following  focuses  on  the  most  closely  related  work  not  covered  by  the  intro¬ 
duction  section.  Existing  approaches  for  CPS  testing  mostly  aim  at  generating 
test  cases  for  existing  models  (e.g.,  [26, 18])  and  do  not  target  testing  of  CPS  tool 
chains.  Code  generator  testing  ([1,  27])  only  target  a  relatively  small  component 
of  the  CPS  tool  chain  but  not  an  entire  CPS  tool  chain. 

Most  of  the  compiler  fuzzers  perform  random  walks  over  a  context-free  gram¬ 
mar,  thus  mainly  focusing  on  generating  syntactically  valid  [14]  and  well  typed 
programs  in  imperative  languages  [28, 10, 11,  29].  None  of  the  works  target  data¬ 
flow  languages  like  Simulink.  We  find  Csmitlr  most  related  to  our  work,  which  is 
state-of-the-art  C  compiler  fuzzer.  Csmith  leverages  the  well-published  C99  stan¬ 
dard  and  can  be  used  to  test  only  a  component  of  entire  CPS  tool  chain  [10].  Our 
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test  generation  and  comparison  techniques  differ  fundamentally  from  Csmith. 
Conceptually,  CPS  tool  chain  fuzzing  is  a  super-set  of  the  schemes  presented  in 
Csmith.  CPS  tool  chains  typically  contain  a  C  compiler;  thus  CyFuzz  leverages 
Csmith  as  a  component. 

Earlier  work  includes  a  differential  testing  based  runtime  verification  frame¬ 
work,  leveraging  a  random  hybrid  automata  generator  [30,25].  Other  works  at¬ 
tack  code  generators  used  in  CPS  tool  chain.  Sturmer  et  al.  generate  model  tak¬ 
ing  specification  of  a  code  generator’s  optimization  rules  in  graph  grammar  [1], 
But  such  specifications  for  code  generators  might  not  be  available  and  white- 
box  testing  in  parts  is  undesirable  [31].  Sampath  et  al.  propose  testing  model¬ 
processing  tools  taking  semantic  nreta-model  of  Stateflow  (a  Simulink  compo¬ 
nent)  [31].  But  the  approach  does  not  scale  and  the  complete  specifications  it 
needs  are  not  available.  In  contrast,  we  propose  the  first  fuzz-testing  framework 
to  test  arbitrary  CPS  tool  chains  based  on  feasible  model  generation. 

Many  CPS  model  verification  and  safety  checking  approaches  have  been  pro¬ 
posed  [8,  32],  Recent  work  verifies  existing  SL/Stateflow  (SL/SF)  models  by  gen¬ 
erating  test  inputs  for  these  models  [18, 19].  Alur  et  al.  analyze  generated  sym¬ 
bolic  traces  of  a  SL/SF  model,  and  combine  simulation  and  symbolic  analysis 
for  improving  coverage  of  given  SL/SF  models  [33].  The  Simulink  Code  Inspector 
compares  generated  code  for  a  given  model  based  on  structural  equivalence  and 
traceability  [21].  However  none  of  these  approaches  describe  random  generation 
of  Simulink  models  for  fuzzing  the  CPS  tool  chain. 

9  Conclusions 

This  work  addresses  the  CPS  tool  chain  quality  problem  using  a  differential  test¬ 
ing  scheme.  Existing  work  either  does  not  test  CPS  development  tool  chains  or 
only  tests  small  subsets.  As  CPS  tool  chains  are  actively  developed  and  released, 
formal  specification  based  test  generation  schemes  are  not  suitable  for  fuzzing 
CPS  tool  chains.  Rather,  our  approach  follows  a  simple  model  generation  strat¬ 
egy  applicable  to  arbitrary  CPS  modeling  languages.  Starting  with  a  random 
and  possibly  erroneous  model,  our  generator  fixes  various  errors  in  the  model 
using  diagnostic  information  returned  by  the  system  under  test.  In  our  experi¬ 
ments  a  high  portion  of  the  generated  models  could  thus  be  executed  without 
errors. 

We  also  define  techniques  to  find  bugs  in  CPS  tool  chains  based  on  simulation 
result  comparison.  The  approach  is  effective  as  our  prototype  implementation 
for  Simulink  found  interesting  cases  and  one  bug.  Although  our  model  generator 
is  scalable  and  fully  automatic,  more  work  is  needed  to  systematically  search 
the  huge  space  of  possible  data-flow  models  and  generate  those  models  that  are 
likely  to  find  bugs  in  modern  CPS  development  environments. 
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The  Simplex  Architecture  ensures  the  safe  use  of  an  unverifiable  complex/smart  controller  by  using  it  in 
conjunction  with  a  verified  safety  controller  and  verified  supervisory  controller  (switching  logic).  This  ar¬ 
chitecture  enables  the  safe  use  of  smart,  high-performance,  untrusted,  and  complex  control  algorithms  to 
enable  autonomy  without  requiring  the  smart  controllers  to  be  formally  verified  or  certified.  Simplex  incor¬ 
porates  a  supervisory  controller  that  will  take  over  control  from  the  unverified  complex/smart  controller  if 
it  misbehaves  and  use  a  safety  controller.  The  supervisory  controller  should  (1)  guarantee  the  system  never 
enters  an  unsafe  state  (safety),  but  should  also  (2)  use  the  complex/smart  controller  as  much  as  possible 
(minimize  conservatism).  The  problem  of  precisely  and  correctly  defining  the  switching  logic  of  the  super¬ 
visory  controller  has  previously  been  considered  either  using  a  control-theoretic  optimization  approach,  or 
through  an  offline  hybrid  systems  reachability  computation.  In  this  work,  we  show  that  a  combined  online/of¬ 
fline  approach  that  uses  aspects  of  the  two  earlier  methods  along  with  a  real-time  reachability  computation, 
also  maintains  safety,  but  with  significantly  less  conservatism,  allowing  the  complex  controller  to  be  used 
more  frequently.  We  demonstrate  the  advantages  of  this  unified  approach  on  a  saturated  inverted  pendulum 
system,  where  the  verifiable  region  of  attraction  is  over  twice  as  large  compared  to  the  earlier  approach.  Ad¬ 
ditionally,  to  validate  the  claims  that  the  real-time  reachability  approach  may  be  implemented  on  embedded 
platforms,  we  have  ported  and  conducted  embedded  hardware  studies  using  both  ARM  processors  and  At- 
mel  AVR  microcontrollers.  This  is  the  first  ever  demonstration  of  a  hybrid  systems  reachability  computation 
in  real-time  on  actual  embedded  platforms,  and  required  addressing  significant  technical  challenges. 
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1.  INTRODUCTION 

Modern  cyber-physical  systems  are  large  complex  systems  of  systems,  where  argu¬ 
ments  about  the  behavior  of  the  whole  system  rely  on  guarantees  about  the  individual 
components.  Individual  components,  however,  may  be  designed  using  machine  learn¬ 
ing  methods  such  as  neural  networks  that  are  currently  not  amenable  to  formal  anal¬ 
ysis,  or  the  components  may  simply  be  too  large  and  complex  for  complete  verification. 
As  such  autonomy  is  incorporated  into  these  increasingly  smart  systems  that  have  the 
ability  to  learn  from  their  environments  and  interactions  through  sophisticated  com¬ 
plex/smart  controllers,  approaches  are  necessary  to  provide  guarantees  about  their 
behavior. 

One  approach  to  provide  formally  verified  behavior  despite  the  use  of  unverified, 
complex,  and  smart  control  logic  is  the  Simplex  Architecture  [Sha  2001].  Similar  to 
how  a  driving  instructor’s  car  may  have  two  steering  wheels  and  two  sets  of  brakes, 
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Fig.  1:  The  Simplex  Architecture  pro¬ 
duces  a  verified  system  despite  the  use 
of  an  unverified  complex/smart  con¬ 
troller.  The  decision  module  should 
switch  between  the  controllers  to  pro¬ 
vide  overall  system  safety. 


Fig.  2:  The  LMI  Simplex  design  ap¬ 
proach  uses  switching  logic  based  on 
an  ellipsoid  within  the  system  con¬ 
straints  in  order  to  produce  a  verified 
system. 


a  Simplex  system  contains  two  controllers  and  supervisory  switching  logic.  As  long 
as  the  instructor  intervenes  to  prevent  dangerous  situations,  the  untrusted  student  is 
allowed  to  drive.  Similarly  in  Simplex,  an  unverified  controller  can  actuate  the  system, 
as  long  as  the  verified  one  takes  over  quickly  at  potentially  unsafe  times. 

In  the  Simplex  Architecture,  shown  in  Figure  1,  unverified  control  logic  (the  com¬ 
plex/smart  controller)  is  wrapped  with  a  verified  controller  (the  safety  controller)  and 
switching  logic  (the  decision  module).  The  complex/smart  controller  typically  has  bet¬ 
ter  performance,  or  is  concerned  with  mission  critical  requirements,  whereas  the  safety 
controller  is  designed  with  simplicity  and  provability  in  mind,  and  may  concern  itself 
only  with  safety-critical  aspects.  When  the  system  is  in  danger  of  entering  an  unre¬ 
coverable  state,  the  decision  module  must  switch  control  to  the  safety  controller.  In 
this  way,  the  complex/smart  controller  can  be  used  while  still  maintaining  the  formal 
guarantees  of  the  safety  controller.  The  key  challenge  when  designing  a  system  with 
the  Simplex  Architecture  is  to  properly  create  the  decision  module  logic. 

It  is  easy  to  design  safe  decision  module  switching  logic;  one  can  simply  always  use 
the  safety  controller.  This  is  undesirable,  however,  as  mission-critical  objectives  might 
be  delayed  or  ignored  since  the  complex/smart  controller  is  never  used.  The  key  chal¬ 
lenge,  which  is  the  focus  of  this  paper,  is  to  reduce  the  conservatism  in  the  decision 
module  design.  Control  should  not  be  switched  too  late,  though,  as  the  safety  controller 
may  not  be  able  to  safely  recover  the  system. 

In  earlier  Simplex  designs,  the  switching  logic  was  designed  in  one  of  two  ways. 
From  a  control  theoretic  perspective,  verified  switching  logic  can  be  synthesized  from 
the  solution  of  a  linear  matrix  inequality  (LMI)  along  with  the  system  dynamics  and 
constraints  [Seto  and  Sha  1999].  Alternatively,  approaches  based  on  hybrid  systems 
reachability  can  be  used  to  produce  a  provably  safe  decision  module  [Bak  et  al.  2011]. 
These  earlier  approaches  will  be  reviewed  in  Section  2.  In  this  paper,  we  propose  the 
use  of  a  unified  approach,  where  the  offline  LMI  result  is  combined  with  an  online 
reachability  computation  to  produce  a  significantly  less  conservative  Simplex  system 
that  is  still  safe.  We  elaborate  on  this  approach  and  prove  its  safety  in  Section  3. 

The  proposed  approach  requires  computing  reachability  online  for  short  time  inter¬ 
vals.  Previous  hybrid  systems  reachability  algorithms,  however,  were  not  designed  for 
real-time  computation  and  furthermore  almost  always  require  the  use  of  numerous 
complex  libraries  for  either  performing  simulations  or  for  representing  sets  of  reach¬ 
able  states  as  some  geometric  data  structure  (such  as  support  functions,  polytopes, 
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zonotopes,  symbolic  expressions,  etc.).  For  this  reason,  in  Section  4,  we  propose  a  real¬ 
time  reachability  algorithm  based  on  mixed  face  lifting  [Dang  2000]  that  is  compatible 
with  the  imprecise  computation  model  in  the  real-time  scheduling  literature  [Lin  et  al. 
1987].  Real-time  reachability  has  applications  beyond  Simplex,  and  is  presented  as  a 
general  online  reachability  approach.  Next,  we  evaluate  the  proposed  unified  Simplex 
design  in  Section  5,  on  both  x86  and  embedded  microprocessors.  In  order  to  provide 
a  direct  comparison,  we  use  the  existing  system  model  from  earlier  Simplex  work  of 
an  inverted  pendulum  system  with  saturation.  The  run-time  approach  significantly 
expands  the  space  where  the  complex/smart  controller  may  be  used.  Other  research 
efforts  related  to  Simplex  and  reachability  are  then  presented  in  Section  6,  followed  by 
conclusions  and  directions  for  future  work  in  Section  7. 

2.  BACKGROUND  AND  CONTRIBUTIONS 

There  have  been  several  verified  design  methodologies  for  systems  that  use  the  Sim¬ 
plex  Architecture.  Before  going  into  their  details,  we  first  present  useful  definitions. 

The  system  is  defined  with  a  set  of  operational  constraints,  such  as  limits  of  ac¬ 
tuators,  physical  restrictions,  invariant  safety  properties  that  cannot  be  violated,  or 
linearization  boundaries  where  the  model  is  considered  valid. 

DEFINITION  1.  States  that  do  not  violate  any  of  the  operational  constraints  are 
called  admissible  states.  Those  that  violate  the  constraints  are  called  inadmissible 
states. 

From  this  definition,  we  can  define  the  set  of  states  that  are  recoverable  for  a  particular 
control  strategy,  assumed  to  be  a  given  safety  controller  in  the  Simplex  architecture. 

DEFINITION  2.  The  set  of  recoverable  states  is  a  subset  of  the  admissible  states, 
such  that  if  the  given  safety  controller  is  used  from  these  states,  all  future  states  will 
remain  admissible. 

The  recoverable  states  are  used  in  the  switching  rule  instead  of  the  admissible  states 
due  effectively  to  inertia  in  the  system.  That  is,  they  are  used  to  ensure  that  the  safety 
controller  and  actuators  have  enough  time  to  prevent  the  system  from  leaving  the 
admissible  states.  Further,  the  intuition  of  defining  the  recoverable  states  as  a  subset 
of  the  admissible  states  is  as  follows.  To  enhance  performance,  we  wish  to  stay  within 
a  small  subset  of  highly  desirable  admissible  states.  The  set  of  recoverable  states  is 
the  subset  of  the  set  of  admissible  states  that  a  safety  control  is  guaranteed  not  to 
leave.  However,  the  safety  controller  may  not  be  able  to  keep  the  system  inside  the 
subset  of  recoverable  states,  namely  the  desirable  states,  and  hence  the  complex/smart 
controller  is  needed.  Their  relation  is  illustrated  in  Figure  2,  where  the  white  ellipsoid 
is  the  recovery  set. 

With  these  definitions,  we  now  describe  two  earlier  approaches  for  verified  Simplex 
design.  The  first  is  based  on  solving  linear  matrix  inequalities  (LMIs),  and  the  second 
is  based  on  reachability  analysis  of  hybrid  systems. 

2.1.  Verified  Design  using  LMIs 

The  first  proposed  way  to  design  a  verified  decision  module  is  based  on  solving  linear 
matrix  inequalities  (LMIs)  [Seto  and  Sha  1999;  Boyd  et  al.  1994],  which  has  been  used 
to  design  Simplex  systems  as  complicated  as  automated  landing  maneuvers  for  an  F- 
16  [Seto  et  al.  1999].  In  this  approach,  system  dynamics  are  approximated  by  a  linear 
model  using  the  standard  control-theoretic  approach,  where  x  =  Ax  +  Bu  for  state 
vector  x  and  input  u. 

In  this  approach,  the  operational  constraints,  as  well  as  saturation  limits  are  ex¬ 
pressed  as  linear  constraints  in  an  LMI.  These  constraints,  along  with  linear  dynam- 
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ics  for  the  system  are  input  into  a  convex  optimization  problem  that  produces  both 
linear  proportional  controller  gains  K  as  well  as  a  positive-definite  matrix  P.  The  con¬ 
troller  produced  is  a  linear-state  feedback  controller,  u  =  Kx,  yielding  the  closed-loop 
dynamics  x  =  (A  +  BK)x.  Given  state  x,  when  input  Kx  is  used,  the  P  matrix  defines 
a  Lyapunov  potential  function  (xTPx)  which  is  positive-definite  with  negative-definite 
derivative  (so  it  is  monotonically  decreasing  over  time),  thus  guaranteeing  stability  of 
the  linear  system  using  Lyapunov’s  direct  or  indirect  (if  the  plant  is  nonlinear  and  was 
linearized)  methods.  Furthermore,  the  matrix  P  is  constructed  by  the  method  such 
that  it  defines  an  ellipsoid  in  the  state  space  where  all  the  constraints  are  satisfied 
when  xTPx  <  1.  Since  the  states  where  saturation  occurs  were  used  as  constraints 
in  the  method,  any  states  inside  the  ellipsoid  result  in  control  commands  that  are  not 
beyond  the  actuator  limits  (where  saturation  would  occur). 

In  this  way,  when  the  gains  K  define  the  safety  controller,  the  ellipsoid  of  states 
xTPx  <  1  is  a  subset  of  the  recoverable  states.  The  situation  is  shown  visually  in  Fig¬ 
ure  2.  The  feasible  region  is  a  subset  of  the  admissible  states  defined  by  the  input 
constraints  (saturation),  as  well  as  the  operational  constraints.  The  stabilizable  region 
(also  known  as  the  region  of  attraction)  is  the  region  of  the  state-space  within  which  a 
given  controller  can  stabilize  the  system.  For  the  purpose  of  LMI-Simplex,  this  is  also 
known  as  the  recoverable  region  or  the  recoverable  states  as  defined  in  Definition  2. 
For  linear  systems  with  constraints,  this  region  may  be  under-approximated  by  solving 
an  LMI  of  the  determinant  maximization  form  [Vandenberghe  et  al.  1998].  For  a  ma¬ 
trix  that  describes  an  ellipsoid  xTPx  =  1,  this  has  the  effect  of  maximizing  the  product 
of  the  radii  of  the  ellipsoid  (which  is  related  to  the  determinant  of  the  matrix  P).  The 
volume  of  an  ellipsoid,  then,  is  proportional  to  this  product.  In  this  way,  the  optimiza¬ 
tion  is  maximizing  the  volume  of  the  ellipsoid  such  that  all  states  inside  do  not  leave 
the  ellipsoid,  and  all  the  constraints  are  satisfied  for  every  state  in  the  ellipsoid. 

This  approach  is  used  to  determine  the  proper  behavior  of  the  decision  module.  As 
long  as  the  system  remains  inside  the  ellipsoid,  any  unverified,  complex/smart  con¬ 
troller  can  be  used.  If  the  state  approaches  the  boundary  of  the  ellipsoid,  control  can 
be  switched  to  the  safety  controller  that  will  drive  the  system  towards  the  equilibrium 
point  where  xTPx  =  0.  Care  must  be  taken  to  ensure  control  is  switched  to  the  safety 
controller  before  the  state  leaves  the  ellipsoid.  If  the  decision  module  simply  checks 
the  Lyapunov  potential  of  the  current  state,  then,  once  the  state  is  outside  of  the  ellip¬ 
soid,  the  system  is  not  guaranteed  to  be  recoverable  without  violating  the  operational 
constraints.  Thus,  a  smaller  subset  of  the  state  space  must  be  used  to  define  the  states 
where  the  complex  controller  is  allowed  to  actuate  the  system.  In  Figure  2,  the  distance 
d  defines  this  extra  buffer  that  can  be  determined  offline  by  computing  the  maximum 
gradient  for  any  control  command  inside  the  ellipsoid,  multiplied  by  the  period  of  the 
decision  logic.  As  long  as  d  is  no  smaller  than  the  maximum  distance  traveled  in  the 
state-space  over  the  time  of  one  full  control  period,  then  d  is  large  enough  to  ensure 
switching  to  the  safety  controller  can  recover  the  system. 

For  safety  it  is  sufficient  to  consider  only  a  single  switch  to  the  safety  controller  and 
never  switching  back.  If  switching  back  is  desired,  this  should  not  be  done  arbitrarily 
as  the  composed  switched  system  might  be  unstable.  Specifically,  the  safety  controller 
should  be  used  at  least  until  a  state  within  the  complex/smart  controller  region  (as 
shown  in  Figure  2)  is  reentered,  before  switching  back  to  the  complex/smart  controller. 

2.2.  Verified  Design  using  Reachability 

An  alternative  method  for  verified  Simplex  design  is  based  on  reachability  analysis 
of  hybrid  systems  [Bak  2013b],  which  has  been  used,  for  example,  to  create  a  Sim¬ 
plex  system  to  prevent  off-road  vehicle  rollover  [Bak  et  al.  2010].  In  this  approach,  the 
dynamics  are  defined  using  a  hybrid  automaton,  which  is  a  formal  model  for  a  sys- 
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tem  with  both  continuous  and  discrete  behaviors.  Mathematically,  a  hybrid  automa¬ 
ton  [Alur  et  al.  1995]  is  a  tuple,  H  =  (A,  L,  X0.  J,  F,  T ),  where: 

—  A  is  the  set  of  continuous  states.  For  a  system  with  n  real-valued  dimensions,  the 
continuous  state  is  R". 

—  L  is  the  set  of  discrete  states  (locations).  The  state  of  a  hybrid  automaton  is  an 
element  of  X  =  L  x  X. 

—  A^0  is  a  set  of  initial  states,  which  is  a  subset  of  X. 

—  I  is  a  set  of  invariants  that  defines  the  continuous  states  that  are  possible  for  each 
location.  It  is  a  function  L  — >  2X . 

—  F  is  a  set  of  flows,  each  of  which  defines  the  differential  equations  in  each  location. 
It  is  a  function  X  — »  2K" . 

—  T  is  a  set  of  discrete  transitions,  each  of  which  defines  switching  between  discrete 
locations.  A  transition  is  composed  of  a  guard  condition  for  when  the  transition  is 
enabled,  and  a  reset  map  that  can  reassign  the  continuous  states  from  the  prede¬ 
cessor  mode  to  the  successor  mode.  In  general,  it  is  a  relation  T  C  X  x  X. 

Semantically,  a  hybrid  automaton  behaves  by  advancing  time  according  to  the  differ¬ 
ential  equations  defined  in  the  mode  of  the  current  discrete  state  l  £  L,  then  allowing 
any  enabled  transitions  to  be  taken,  and  repeating,  yielding  a  sequence  of  states  called 
an  execution.  A  state  £  £  X  is  reachable  is  there  exists  a  finite  execution  ending  in 
x.  The  set  of  reachable  states  contains  every  reachable  state.  The  guard  conditions  on 
the  outgoing  transitions  define  when  the  location  can  change.  The  invariants  of  the 
locations  can  be  used  to  force  transitions  by  preventing  time  from  elapsing  further  in 
the  current  mode.  Together,  these  allow  nondeterminsim  in  the  discrete  behavior.  A 
hybrid  automaton  can  be  graphically  depicted  as  a  finite-state  machine  with  differen¬ 
tial  equations  in  each  discrete  state.  The  model  also  allows  for  nondeterminism  in  the 
continuous  behavior  because  a  single  state  x  £  X  may  be  associated  a  set  of  derivatives 
for  each  variable,  via  the  set  of  flows  F. 

This  modeling  framework  is  very  expressive,  and  computing  exactly  the  sets  of  states 
a  hybrid  automaton  may  enter,  called  the  reachable  set  of  states,  is  undecidable  [Hen- 
zinger  et  al.  1995].  Thus,  analysis  of  hybrid  systems  often  restricts  either  the  contin¬ 
uous  dynamics  or  the  discrete  dynamics  [Alur  and  Dill  1994;  Lafferriere  et  al.  2000; 
Branicky  1998].  In  this  paper,  the  reachability  algorithm  proposed  in  Section  4  con¬ 
siders  restricted  hybrid  automata  models  where  (a)  the  state  invariants  are  disjoint 
and  cover  the  continuous  states  KT,  (b)  there  are  no  reset  maps  in  the  transitions  be¬ 
tween  discrete  states,  and  (c)  the  guards  of  incoming  transitions  are  defined  by  the 
state  invariants. 

In  addition  to  restrictions  on  dynamics,  practical  reachability  approaches  often  over¬ 
approximate  the  set  of  reachable  states  [Kapinski  and  Krogh  2002;  Dang  et  al.  2010; 
Frehse  et  al.  2011],  which  is  sufficient  for  proving  safety  properties.  If  a  sound  over¬ 
approximation  of  the  reachable  set  of  states  for  a  hybrid  automaton  does  not  contain 
any  unsafe  states,  then  the  system  is  verified  as  safe  since  no  unsafe  states  are  in  the 
actual  reachable  set  of  states  either.  That  is,  the  system  is  safe  if  the  intersection  of  the 
over-approximation  of  the  set  of  reachable  states  and  the  set  of  unsafe  states  is  empty. 
This  approach  may,  however,  lead  to  spurious  counterexamples  where  the  error  due  to 
the  over-approximation  contains  unsafe  states,  but  the  actual  reachable  set  of  states 
does  not. 

We  define  REACH,*,  (x,  HA)  to  be  the  set  of  states  reached  in  any  amount  of  time  from 
state  x  in  hybrid  automaton  HA,  Reaoh<,(.x',  HA)  is  the  set  of  states  reached  from  x  in 
up  to  t  time,  and  REACH=t(a;,  HA)  is  the  set  of  states  reached  after  exactly  t  time  has 
elapsed.  Also,  we  naturally  extend  Reach  to  initial  sets  of  states,  where  the  resultant 
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set  of  reachable  states  is  the  union  of  the  set  of  reachable  states  from  each  state  in  the 
initial  set. 

In  terms  of  Simplex  design,  the  behavior  of  an  optimal  decision  module  can  be  de¬ 
fined  in  terms  of  reachability.  Optimal  here  means  that  the  given  safety  controller 
takes  over  only  if  it  has  to;  if  it  did  not  take  over,  then  the  system  remains  in  admis¬ 
sible  states  and  can  enter  the  subset  of  recoverable  states  that  can  be  pre-computed 
offline  e.g.,  using  LMIs.  Furthermore,  it  never  takes  over  when  the  complex/smart 
controller  could  safely  be  used.  The  switching  condition  (formalized  as  the  transition’s 
guard  and  invariant  in  the  hybrid  automaton)  between  the  safety  controller  and  com¬ 
plex/smart  controller  modes  is  defined  using  the  following  theorem  [Bak  et  al.  2011]. 

THEOREM  3.  The  optimal  switching  condition  for  Simplex  is  given  when,  at 
every  control  iteration,  the  complex  I  smart  controller  is  used  if  and  only  if 
(1)  REACH<5(a:,CC)  n  U  =  0  and  (2)  REACH00(REACH=5(j:1  CC),  SC)  n  U  =  0,  where 
x  £  X  is  the  current  state  and  U  C  X  is  the  set  of  inadmissible  ( unsafe)  states. 

The  inner  REACH=^  in  part  (2)  is  the  time-bounded  reachability  of  the  system  for  one 
decision  logic  switching  interval  time,  5,  while  using  the  complex/smart  controller  (CC). 
The  outer  REACH,*,  is  the  infinite-time  reachability  for  the  system  under  control  of  the 
safety  controller  (SC). 

Intuitively,  this  check  is  examining  what  happens  if  the  complex/smart  controller 
is  used  for  a  single  control  interval  of  time  S,  and  then  the  safety  controller  is  used 
thereafter.  If  this  set  of  states  contains  an  inadmissible  state  (either  before  the  switch 
as  in  part  (1)  or  after  as  in  part  (2)),  then  the  complex/smart  controller  cannot  be 
used  for  one  more  control  interval,  and  instead  the  safety  controller  must  be  used 
right  away.  Assuming  the  system  starts  in  a  recoverable  state,  this  guarantees  it  will 
remain  in  the  recoverable  set  for  all  time. 

Several  factors  prevent  the  direct  use  of  Theorem  3.  The  first  is  that  the  reason  to 
apply  Simplex  is  that  a  precise  model  of  the  complex/smart  controller  is  not  available, 
but  rather  an  over-approximation  must  be  used  which  can  be  computed,  for  exam¬ 
ple,  based  on  the  plant  model  and  actuator  limits.  Second,  as  discussed  before,  com¬ 
puting  reachability  exactly  for  a  general  hybrid  automaton  is  undecidable.  However, 
estimates  of  the  set  of  recoverable  states  (Definition  2),  can  still  be  computed  using 
over-approximations,  where  the  conservativeness  of  the  resultant  decision  module  de¬ 
pends  on  the  amount  of  over-approximation.  Third,  the  switching  condition  is  defined 
in  terms  of  a  specific  state  x,  which  is  not  useful  for  offline  computation  since  every 
state  would  need  to  be  enumerated.  Instead,  the  condition  can  be  rewritten  in  terms 
of  backwards  reachability  from  the  set  of  inadmissible  states,  which  can  then  be  com¬ 
puted  offline  [Bak  et  al.  2011;  Bak  2013b].  As  with  the  LMI  approach,  the  output  is  a 
set  of  states  which  forms  a  guaranteed  subset  of  the  recoverable  states.1.  These  con¬ 
siderations  are  combined  in  order  to  provide  a  condition  for  effectively  computing  the 
decision  module  logic  as  follows. 

COROLLARY  4.  A  safe  switching  condition  for  Simplex  is  given  when,  at  every 
control  iteration,  the  complex  /  smart  controller  is  used  if  the  current  state  x  <f 
BACKREACH^  (BACKREACH*  (£/,  SC),  CC'). 

Here,  BackReach*  is  an  over-approximation  of  the  exact  set  of  backward  reachable 
states  for  all  time  (that  is,  to  a  fixed-point).  The  inner  BackReach*  defines  the  states 


1The  set  of  backward  reachable  states  can  be  computed  for  deterministic  systems  (including  linear  systems) 
by  negating  the  differential  equations  and  inverting  the  transitions  in  the  hybrid  automaton  and  using 
standard  forward  reachability  techniques.  This  technique  is  known  as  back-reachability. 
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where,  if  the  system  were  to  start  from  the  set  of  unsafe  states  U  and  use  the  safety 
controller,  it  could  still  violate  one  or  more  of  the  safety  constraints.  Then,  the  outer 
BackReach<5  is  the  set  of  states  that,  within  one  control  interval,  can  reach  an  un¬ 
recoverable  state.  Since  the  unrecoverable  states  contain  the  unsafe  states,  and  since 
the  outer  BackReach'<7)  checks  up  to  S  time  rather  than  exactly  5  time,  a  separate 
condition  is  not  needed  to  check  if  the  the  complex/smart  controller  itself  reaches  the 
unsafe  states,  as  in  part  (1)  of  Theorem  3. 

The  pessimism  in  the  resultant  decision  logic  depends  on  both  the  accuracy  of  the 
reachability  computation  as  well  as  on  how  much  CC'  over-approximates  the  exact 
complex/smart  controller  model  CC.  The  condition  in  Corollary  4  is  more  useful  than 
the  one  in  Theorem  3  because  it  can  be  effectively  computed  using  existing  hybrid  sys¬ 
tems  reachability  algorithms.  The  set  of  states  on  the  right-hand  side  can  be  computed 
offline  and  encoded  in  some  form  (for  example,  using  linear  bounds  [Bak  2009])  and 
then,  online,  the  decision  module  need  only  check  if  the  current  state  exists  within  the 
encoded  set  of  states.  If  it  does,  then  the  safety  controller  must  be  immediately  used. 
If  it  does  not,  then  the  complex/smart  controller  can  be  used  for  one  control  interval, 
after  which  the  condition  will  be  checked  again  on  the  new  state. 

2.3.  Contributions 

In  this  paper,  building  on  our  prior  work  [Bak  et  al.  2014],  we  show  how  to  combine 
the  LMI-based  Simplex  method  with  a  real-time  reachability  method  into  a  unified 
framework  to  ensure  safety  while  drastically  decreasing  the  overconservative  use  of 
the  safety  controller.  Specifically,  if  it  is  possible  to  use  the  set  of  recoverable  states 
computed  using  the  LMI  method  for  the  switching  condition,  we  do  so.  If  not  and 
the  system  is  at  a  state  outside  the  recoverable  states  based  on  the  LMI  ellipsoids, 
then  we  try  to  check  safety  using  a  novel  real-time  reachability  method,  in  contrast  to 
the  previous  offline  reachability  approach.  Together,  we  illustrate  how  this  unified  ap¬ 
proach  gives  both  real-time  guarantees  and  reduces  conservatism  of  when  the  safety 
controller  is  used.  A  main  contribution  of  our  approach  is  the  first  ever  demonstra¬ 
tion  of  a  reachability  method  in  real-time,  enabled  by  our  careful  design  and  imple¬ 
mentation  that  does  not  use  any  dynamic  memory  allocation  nor  rely  on  sophisticated 
(non-portable)  libraries  that  many  other  methods  use,  such  as  the  Parma  Polyhedral 
Library  (PPL)  [Bagnara  et  al.  2008],  recent  satisfiability -modulo  theories  (SMT)  ap¬ 
proaches  [Gao  et  al.  2013],  or  validated  integration  tools  [Duggirala  et  al.  2013].  To 
validate  the  feasibility  of  actually  implementing  the  method  in  real-time  embedded 
hardware,  we  have  ported  our  prototype  method  from  [Bak  et  al.  2014]  that  was  im¬ 
plemented  on  x86-64  platforms  to  several  embedded  platforms  (namely  a  32-bit  ARM- 
based  system  and  an  8-bit  Atmel  AVR  ATmega32u4-based  Arduino  system).  This  ef¬ 
fort  validates  our  claims  from  [Bak  et  al.  2014],  which  were  not  previously  validated  in 
embedded  hardware.  The  key  result  of  this  paper  is  the  first  ever  demonstration  of  a  hy¬ 
brid  systems  reachability  algorithm  implemented  in  embedded  hardware  that  can  meet 
real-time  guarantees,  which  required  carefully  designing  the  reachability  algorithm  as 
described  in  this  paper.  We  have  additionally  added  significant  further  details  of  the 
approach  and  case  study  to  the  paper  over  [Bak  et  al.  2014],  including  code  snippet 
examples  for  the  case  study. 

3.  UNIFIED  APPROACH  FOR  SIMPLEX  DESIGN 

The  two  existing  approaches  for  Simplex  design  previously  discussed  each  have  their 
own  limitations.  The  LMI  approach  works  when  the  system  model  is  linear.  If  there 
are  actuator  limits,  and  the  input  to  the  actuators  u  (from  x  =  Ax  +  Bu)  can  saturate, 
the  output  of  the  optimization  will  be  a  set  of  states  where  the  command  used  by  the 
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safety  controller  is  within  the  saturation  limits.  This  is  done  by  adding  a  constraint 
based  on  the  state-feedback  gain  as  part  of  the  optimization  (the  input  is  u  =  Kx, 
which  is  bounded  by  the  linear  constraints  Kx  <  MAX_INPUT  and  Kx  >  MIN_INPUT). 

The  set  of  states  output  by  the  LMI  approach  is  safe,  but  may  be  pessimistic,  since 
a  saturated  safety  controller  may  still  be  able  to  recover  the  system.  Furthermore, 
the  resultant  switching  condition  is  based  on  a  Lyapunov  function  which — due  to  con¬ 
vexity  and  quadratic  restrictions  required  in  the  optimization  algorithms — has  level 
sets  that  are  ellipsoidal.  This  is  a  sufficient  but  not  necessary  condition  for  stability 
and  therefore  the  switching  set  is  almost  certainly  conservative.  We  demonstrate  this 
pessimism  in  our  evaluation  in  Section  5. 

The  reachability-based  Simplex  approach  is  not  restricted  to  linear  systems,  and  can 
have  its  conservatism  decreased  by  increasing  the  accuracy  of  the  reachability  compu¬ 
tation2.  One  downside  of  this  approach  is  that  over-approximation  error  occurs  from 
the  need  to  abstract  the  complex/smart  controller  hybrid  automaton  by  a  hybrid  au¬ 
tomaton  which  takes  into  account  any  possible  complex/smart  controller  command.  A 
second  issue  is  the  difficulty  of  succinctly  and  accurately  encoding  the  result  of  the  com¬ 
putation,  which  in  general  may  be  a  large  non-convex  set  in  many  dimensions.  Lastly, 
hybrid  systems  reachability  methods  introduce  over-approximation  error,  which  can 
be  large  when  the  initial  set  of  states  is  large  and  the  reachability  time  bound  is  large. 
The  back-reachability  formulation  of  Theorem  3  includes  a  time-unbounded  reacha¬ 
bility  computation  from  the  set  of  inadmissible  states,  which  can  require  significant 
computation  time. 

We  now  present  an  alternative  design  for  a  verified  Simplex  system.  The  proposed 
technique  makes  use  of  aspects  from  both  of  the  previous  verified  design  approaches 
in  order  to  overcome  some  of  their  individual  limitations. 

First,  we  formalize  the  connection  of  the  ellipsoid  from  of  the  LMI  approach  with 
that  of  a  reachability  computation  of  a  hybrid  automaton  (which  by  the  ellipsoid’s  con¬ 
struction  remains  in  a  single,  unsaturated  mode): 

LEMMA  5.  The  output  of  the  LMI  approach,  the  potential  function  P  and  controller 
gains  K,  define  a  safety  controller  SC  and  a  subset  of  the  recoverable  set  of  states  R  = 
{x\xT Px  <  1},  where  REACH00(7^,  SC)  fl  U  =  0. 

This  is  true  because  the  potential  function  is  guaranteed  to  satisfy  the  constraints 
passed  to  the  LMI  solver,  including  avoidance  of  the  inadmissible  states,  when 
XTPX  <  1.  When  the  controller  gain  vector  K  output  by  the  approach  is  used  (which 
defines  the  safety  controller  update  u  =  Kx),  the  potential  function  is  strictly  decreas¬ 
ing  over  time  (i.e.,  it  is  a  Lyapunov  function).  Therefore,  it  is  guaranteed  for  unbounded 
time  that  any  state  starting  inside  R  will  remain  inside  R.  Since  there  are  no  inad¬ 
missible  states  in  R,  no  inadmissible  states  will  ever  be  reached. 

We  can  now  define  an  alternate  condition  for  safe  switching  logic: 

THEOREM  6.  A  safe  switching  condition  for  Simplex  is  given  when,  at  ev¬ 
ery  control  iteration,  the  complex  I  smart  controller  is  used  if,  for  some  a  time, 
(1)  Reaches,  CC)  n  U  =  0,  (2)  REACH<a(REACH=,5(:r,CC),SC)  n  U  =  0,  and 
(3)  Reach=C[(Reach=5(x,  CC),  sc)  c  n. 

PROOF.  Intuitively,  this  switching  condition  states  that  the  complex/smart  con¬ 
troller  may  be  used  if:  (1)  the  complex/smart  controller  cannot  reach  an  unsafe  state 
before  the  next  decision  interval  (at  time  <5),  (2)  if  the  safety  controller  takes  over  at 
the  next  decision  interval,  it  will  avoid  unsafe  states  until  S  +  a  times  passes,  and  (3) 
after  S  +  a  time,  a  state  in  R  will  be  reached. 


2  Over-approximating  reachability  approaches  typically  have  an  accuracy  /  computation  time  trade  off. 
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More  formally,  assume  by  contradiction  that  this  is  not  a  safe  switching  condition, 
so  an  inadmissible  state  is  reached  at  some  time.  This  time  will  be  either  less  than  <5, 
more  than  S  and  less  than  <5  +  a,  or  more  than  S  +  a.  The  first  two  of  these  cases  are 
ruled  out  directly  by  conditions  (1)  and  (2),  so  only  the  third  case  needs  to  be  examined. 

From  Lemma  5  REACH00(^, SC)  n  U  =  0.  If  ft'  C  7^,  REACH00(7e', SC)  C 
REACH00(7?.,  SC),  then  the  smaller  set  of  states  7 Z'  =  REACH=Q(REACH=,5(x,  CC),  SC)  C 
TZ  will  also  satisfy  the  condition  REACH (7?/.  SC)  DC/  =  0.  Therefore,  every  state  reached 
after  5  +  a  is  also  admissible. 

Since  all  three  cases  do  not  contain  an  inadmissible  state,  our  assumption  that  an 
inadmissible  state  is  reached  is  violated,  yielding  a  contradiction,  and  therefore  this  is 
a  safe  switching  condition.  □ 

In  summary,  the  proposed  approach  is  as  follows:  when  the  system  is  well-inside  the 
ellipsoid  that  represents  the  largest  safe  sublevel  set  of  the  Lyapunov  function,  we  do 
not  need  to  invoke  an  extensive  reachability  analysis  using  the  safety  controller,  as 
we  know  the  state  is  recoverable  (even  for  the  next  control  period).  When  the  system 
state  is  near  the  boundary  of  the  ellipsoid,  the  reachability  analysis  is  used  to  allow 
the  system  to  cross  the  boundary  of  the  ellipsoid  as  long  as  the  reachability  compu¬ 
tation  shows  that  (1)  no  system  constraints  are  violated  when  this  is  done  (i.e.,  none 
of  the  reachable  states  violate  a  system  constraint),  and  (2)  the  state  can  be  guaran¬ 
teed  to  be  brought  back  into  the  ellipsoid  (i.e.,  the  reachable  states  return  inside  the 
ellipsoid).  This  allows  the  complex  controller  to  be  used  in  a  larger  region  compared 
with  the  LMI-approach  because  it  can  soundly  reason  about  the  behavior  of  the  system 
outside  of  the  ellipsoid  (remember  that  the  Lyapunov  function  from  the  LMI  method 
is  only  a  sufficient  condition  for  safe  switching).  This  condition  can  also  be  less  conser¬ 
vative  than  the  pure  reachability  approach  because  the  computation  needed  is  from 
a  single  state  x,  rather  than  the  possibly  large  set  of  inadmissible  states.  Addition¬ 
ally,  it  involves  reasoning  over  a  finite-time  horizon  (a  +  <5),  rather  than  infinite-time 
reachability  needed  in  the  method  based  on  Theorem  3. 

There  are  still  two  issues  which  need  to  be  addressed  before  the  condition  in  Theo¬ 
rem  6  is  usable.  First,  since  we  cannot  compute  reachability  exactly  for  complex  hybrid 
automata  due  to  decidability  reasons  [Henzinger  et  al.  1995],  we  will  instead  compute 
an  over-approximation.  This  will  result  in  a  conservative  switching  set  depending  on 
the  accuracy  of  the  computation.  Second,  this  computation  is  defined  from  the  system’s 
current  state  x,  which  is  not  available  offline.  In  order  to  resolve  this  issue,  we  propose 
an  online,  real-time  reachability  computation  method  in  the  next  section.  After  that, 
in  Section  5,  we  will  evaluate  the  conservatism  in  the  switching  set  due  to  the  over¬ 
approximation  in  the  proposed  algorithm. 

4.  REAL-TIME  REACHABILITY  ALGORITHM 

Hybrid  systems  reachability  computations  have  been  traditionally  computed  offline, 
and  are  both  memory  and  processor  intensive  operations.  In  Section  3,  we  have  il¬ 
lustrated  several  reasons  to  perform  the  reachability  computation  at  runtime.  This 
requires  a  reachability  algorithm  capable  of  use  within  a  real-time  system.  In  this  sec¬ 
tion,  we  describe  a  real-time  reachability  algorithm  with  the  following  key  features: 

—  High-performance  for  a  quick  runtime  for  short  reachability  times. 

—  The  ability  to  check  the  three  conditions  from  Theorem  6. 

—  No  dynamic  data  structures  (or  large  memory  preallocation)  or  recursion,  for  us¬ 
ability  in  a  real-time  system. 

—  No  dependence  on  complex  external  libraries  (only  the  C  standard  library)  that  most 
if  not  all  other  reachability  approaches  use. 

—  Iterative  improvement  in  accuracy  with  increased  computation  time. 
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The  last  point  is  important  because  it  allows  the  reachability  task  to  be  scheduled  in 
the  framework  of  imprecise  real-time  system  computation  [Liu  et  al.  1994].  In  this 
framework,  each  task  produces  a  partial  result  that  is  usable  and  improved  upon  as 
more  computation  time  is  added  (this  is  sometimes  called  an  anytime  algorithm).  In 
particular,  the  proposed  reachability  algorithm  is  based  on  the  milestone  approach  [Lin 
et  al.  1987],  where  partial  results  are  recorded  at  various  points  during  the  execution, 
and  the  last-recorded  values  are  used  when  the  final  result  is  needed.  This  is  in  con¬ 
trast  to  the  traditional  real-time  systems  execution  model  where  each  task  has  a  fixed 
worst-case  execution  time  (WCET)  [Liu  and  Layland  1973]. 

We  now  present  the  real-time  reachability  algorithm  that  is  suitable  for  real-time, 
online,  computation  that  satisfies  the  above  requirements.  We  distinguish  between 
reach-time,  which  is  the  time  we  are  computing  reachability  for,  and  runtime,  which 
is  the  duration  of  (wall)  time  the  method  is  allowed  to  run.  Recall  that  the  types  of 
hybrid  systems  we  consider  are  ones  where  the  state  invariants  are  disjoint  and  cover 
the  continuous  state  KT,  there  are  no  reset  maps  in  the  transitions  between  discrete 
states,  and  the  guards  of  incoming  transitions  are  defined  by  the  state  invariants.  In 
these  piecewise  systems,  the  state  of  the  hybrid  automaton  can  be  determined  solely 
by  the  continuous  state,  although  different  differential  equations  can  be  used  in  dif¬ 
ferent  parts  of  the  state  space.  This  is  applicable  to  many  state-feedback  continuous 
systems  with  saturation  (such  as  those  using  gain  scheduling  controllers)  since  the 
states  where  saturation  occurs  are  typically  disjoint  from  the  unsaturated  states  (be¬ 
cause  the  actuator  command  is  a  function  of  the  state),  and  the  continuous  states  do 
not  jump  along  the  saturation  boundary. 

To  employ  the  real-time  reachability  algorithm,  as  in  our  earlier  work  [Bak  et  al. 
2011],  the  user  defines  the  system  dynamics  through  a  function  (a  function  written 
in  the  C  language  in  this  implementation)  that  returns  the  minimum  and  maximum 
derivative  in  each  dimension  given  an  arbitrary  box  of  the  state  space.  The  derivative 
needed  in  the  algorithm  is  always  in  the  outward  direction  of  the  box  of  states  being 
tracked.  The  tracked  box  has  2 n  faces,  where  n  is  the  number  of  dimensions.  For  each 
of  the  n  dimensions,  these  faces  are  represented  by  a  minimum  face,  and  a  maximum 
face.  That  is,  there  are  total  2 n  minimum  and  maximum  faces,  each  of  which  refers  to 
particular  faces  of  a  hyperrectangle  used  to  represent  portions  of  the  set  of  reachable 
states.  If  the  minimum  face  is  being  considered,  the  minimum  of  the  derivative  is  used, 
as  this  may  (but  not  necessarily  so)  push  the  tracked  states  outward  from  the  hyper¬ 
rectangle.  If  the  maximum  face  is  being  considered,  the  maximum  of  the  derivative 
is  used  for  the  same  reason.  Nonlinear  dynamics  are  permitted  in  this  approach,  so 
long  as  the  user-provided  function  maximizes  or  minimizes  the  nonlinear  derivatives 
within  an  arbitrary  box.  Notice  that  this  does  not  require  solving  the  differential  equa¬ 
tions  (which  is  generally  a  harder  problem),  since  the  bounds  are  on  the  derivatives 
themselves.  Furthermore,  we  require  the  derivatives  are  defined  in  the  entire  state 
space,  and  that  they  are  bounded. 

The  real-time  reachability  algorithm  is  based  on  mixed  face  lifting  [Dang  and  Maler 
1998;  Dang  2000].  This  approach  is  a  flow-pipe  construction  method,  which  means  that 
snapshots  of  the  reachable  set  of  states  are  computed  at  increasing  points  in  reach¬ 
time,  and  reasoning  is  done  about  which  states  can  be  encountered  between  snapshots. 

To  create  a  real-time  implementation,  we  use  boxes  (n-dimensional  hyper¬ 
rectangles)  as  our  representation  of  the  set  of  states.  Over  long  reach-times,  this  repre¬ 
sentation  can  be  problematic  because,  if  the  actual  reachable  set  of  states  is  not  a  box, 
error  is  introduced  by  over-approximating  it  as  one  (called  the  wrapping  effect  [Moore 
1966]).  However,  since  we  only  need  to  compute  reachability  for  short  reach-times  (J+q 
from  Theorem  6),  a  simpler,  faster,  representation  is  preferred  to  better  long-term  error 
control.  In  mixed  face  lifting,  the  dynamics  along  each  face  are  over-approximated  by 
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Algorithm  1  The  real-time  reachability  algorithm  uses  a  desired  reach-time  step  to 
tune  its  runtime  based  on  the  available  computation  time. 

1  Box  currentBox  :=  initialBox 

3  while  (reachTimeRemaining  >  0) 

Box[]  nebs  =  constructNeighborhoods ( currentBox  ,  reachTimeStep ) 

5 

crossReachTime  :=  minCrossReachTime(nebs) 

7  advanceReachTime  :=  min(  crossReachTime  ,  reachTimeRemaining) 
currentBox  :=  advanceBoxf  nebs ,  advanceReachTime) 

9 

reachTimeRemaining  :=  reachTimeRemaining  —  reachTimeToAdvance 

1 1  end  while 


the  maximum  derivative  along  that  face.  The  reach-time  is  then  advanced  uniformly 
along  all  faces  (i.e.,  in  all  directions). 

We  modify  the  original  mixed  face  lifting  algorithm  to  make  it  usable  in  a  real¬ 
time  setting.  In  particular,  instead  of  using  the  desired  error  in  order  to  control  the 
neighborhood  width  around  each  face  [Dang  2000],  we  use  a  desired  reach-time  step 
to  control  neighborhood  widths.  This  parameter  allows  us  to  tune  the  total  number 
of  steps  used  in  the  method,  and  therefore  alter  the  runtime.  After  the  given  reach¬ 
time  is  obtained,  the  desired  step  size  is  decreased  (which  reduces  the  width  of  the 
neighborhoods,  and  therefore  the  derivative  error  at  each  step)  and  the  computation  is 
restarted.  In  our  algorithm,  initially  we  use  a  time  step  which  is  some  factor,  say  one 
tenth,  of  the  desired  reach  time.  The  decrease  is  computed  by  dividing  the  time  step 
by  two.  In  this  way,  the  algorithm  will  produce  progressively  more  accurate  answers, 
for  as  much  runtime  as  the  task  is  given. 

The  high-level  algorithm,  given  a  fixed  desired  step  size  (reachTimeStep),  is  given 
in  Algorithm  1.  For  a  box,  there  are  two  faces  for  every  dimension  (one  for  each  of  the 
minimum  and  maximum  faces  along  that  dimension),  and  there  are  two  corresponding 
face  neighborhoods  (regions  where  the  face  may  advance  through  during  the  current 
time  step)  for  every  dimension.  The  neighborhoods,  nebs,  are  constructed  based  on  the 
desired  reach-time  step.  This  neighborhood  construction  process  will  be  elaborated  on 
shortly. 

Next,  the  minimum  reach-time  for  any  point  along  each  face  to  cross  the  correspond¬ 
ing  neighborhood  in  the  corresponding  direction  is  computed.  What  this  means  is  that, 
for  example  in  the  two-dimensional  example  of  Figure  3,  the  minimum  reach-time  for 
any  point  along  the  left  face  of  currentBox  to  cross  to  the  left  side  of  nebs  [  0  ]  in  the 
x  direction  is  computed,  as  well  as  the  minimum  reach-time  for  any  point  along  the 
right  face  to  cross  nebs  [  1  ] ,  as  well  as  the  neighborhoods  in  the  y  directions,  and  then 
the  minimum  of  all  of  these  is  returned.  This  is  computed  by  looking  at  the  minimum 
or  maximum  derivative  within  the  box  for  each  neighborhood  (from  the  user-provided 
derivative  bounds  function),  as  well  as  the  width  of  the  neighborhood  along  the  corre¬ 
sponding  dimension. 

Finally,  the  currentBox  at  the  next  reach-time  step  is  computed  based  on  the  neigh¬ 
borhoods  and  computed  reach-time  to  advance  (which  may  be  reduced  if  it  exceeds 
reachTimeRemaining).  This  is  done  by  advancing  each  face  by  the  maximum  deriva¬ 
tive  in  the  outward  direction  in  its  neighborhood  (from  the  user-provided  derivative 
bounds  function)  multiplied  by  advanceReachTime. 

The  novel  aspect  of  this  face  lifting  reachability  algorithm  is  that  the  widths  of  the 
neighborhoods  are  tunable  by  the  reachTimeStep  parameter.  The  neighborhood  con¬ 
struction  (the  constructNeighborhoods  function)  proceeds  in  three  steps: 
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Fig.  3:  The  neighborhood 
widths  are  determined  by 

reachTimeStep  and  the 

derivatives  along  the  faces  of 

currentBox. 


Fig.  4:  Although  the  derivative  along  the  face 
may  be  inward-facing,  the  derivative  in  the 
neighborhood  can  still  be  outward  facing.  The 
first  condition  of  step  3  in  the  neighborhood 
construction  process  checks  for  this  and  recon¬ 
structs  the  neighborhoods  if  such  a  situation  oc¬ 
curs.  Here,  nebs  [1]  would  be  updated  to  an 
outward-facing  neighborhood,  which  would  re¬ 
quire  subsequent  reconstruction  of  the  other 
neighborhoods  (because  the  edges  overlap). 


(1)  The  maximum  outward  derivative  along  each  face  of  currentBox  is  computed.  One 
neighborhood  is  constructed  for  each  face,  where  the  width  of  the  corresponding 
neighborhood  is  based  on  the  derivative  (the  width  is  the  derivative  multiplied  by 
the  passed-in  desired  reachTimeStep). 

(2)  The  neighborhood  boxes  are  all  constructed  based  on  the  computed  widths,  such 
that  the  edges  overlap  as  shown  in  Figure  3.  We  call  a  neighborhood  constructed 
on  the  inside  of  the  corresponding  face  an  inward-facing  neighborhood  (such  as 
nebs  [  1  ]  in  the  figure). 

(3)  The  outward  derivatives  in  the  constructed  neighborhoods  are  computed  with  the 
user-provided  function.  If  either  (1)  an  inward-facing  neighborhood  contains  an 
outward-facing  derivative,  or  (2)  a  derivative  has  doubled  in  value  since  the  previ¬ 
ous  derivative  computation  for  that  neighborhood  (which  initially  is  the  flat  neigh¬ 
borhood),  the  width  of  the  neighborhood  is  recomputed  and  the  process  repeats  by 
returning  to  step  2. 

The  check  in  step  3  ensures  two  things.  The  first  condition  is  necessary  in  case  a 
derivative  was  inward-facing  in  a  previously-constructed  neighborhood,  but  outward¬ 
facing  in  the  new,  larger  neighborhood.  This  case  is  shown  visually  in  Figure  4.  The 
second  condition  guarantees  that  the  reach-time  to  progress  from  a  point  on  the  face 
through  the  corresponding  face  neighborhood  is  at  least  reachTimeStep/2.  Due  to 
this,  we  can  bound  the  maximum  number  of  iterations  of  the  while  loop  as  the  desired 
reach  time  divided  by  reachTimeStep/2.  Since  the  edges  of  the  neighborhoods  over¬ 
lap,  the  neighborhoods  of  the  other  faces  need  to  be  reconstructed  as  well,  which  is 
why  the  algorithm  backtracks  to  step  2. 

The  number  of  times  the  neighborhood  construction  backtracks  from  step  3  to  step  2 
is  also  bounded.  This  is  because  a  face  can  flip  from  inward-facing  to  outward-facing 
only  once,  and  since  it  was  assumed  there  is  a  maximum  derivative  in  the  state  space, 
the  observed  derivative  can  only  double  a  finite  number  of  times. 

The  imprecise  computation  version  of  the  algorithm  proceeds  by  running  Algo¬ 
rithm  1  repeatedly,  decreasing  reachTimeStep  after  each  repetition.  In  our  imple- 
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mentation,  after  each  execution  reachTimeStep  was  halved,  although  strategies 
other  than  halving  are  also  possible  (this  is  a  trade  off  between  the  time  between 
milestones  and  the  error  reduction  obtained  at  each  iteration).  When  the  deadline  is 
reached  (or  the  real-time  reachability  task  is  stopped),  the  most  recent  reachability 
result  is  used  as  the  output.  For  this  reason,  the  exact  number  of  iterations  of  the 
neighborhood  construction  loop  is  not  too  useful,  as  long  as  it  has  an  upper  bound,  and 
we  can  adjust  it  with  reachTimeStep. 

If  the  derivative  doubles  several  times,  the  tracked  box  will  be  pessimistic,  since 
the  conservatism  comes  from  over-approximating  a  derivative  in  a  neighborhood  by 
its  maximum  value.  For  this  reason,  we  also  set  a  threshold  in  the  loop  for  how  large 
the  tracked  boxes  are  allowed  to  get  (not  shown),  and  if  it  is  exceeded  we  immediately 
halve  reachTimeStep  and  restart  the  loop.  If  the  number  of  backtracks  to  step  2  is 
small  (which  is  true  in  practice),  each  advancement  of  time  takes  0{n)  where  n  is  the 
number  of  dimensions  in  the  system. 

From  the  four  desired  properties  of  a  real-time  reachability  algorithm  mentioned 
earlier,  this  algorithm  is  quick  (no  exponential  complexity  operations),  requires  no  dy¬ 
namic  memory  or  recursion,  and  can  iteratively  provide  a  better  answer.  In  order  to 
satisfy  the  remaining  desired  condition,  we  need  to  provide  the  ability  to  check  the 
three  conditions  from  Theorem  6.  Rather  than  first  computing  the  reachable  set  of 
states  and  then  checking  the  conditions  in  that  set  (which  would  require  dynamic  stor¬ 
age  to  store  the  reachable  set),  we  instead  modify  the  core  algorithm  in  Algorithm  1 
to  do  the  checks  during  the  computation.  Conditions  (1)  and  (2)  of  the  theorem  deal 
with  the  safety  of  reachable  states  at  intermediate  reach-times.  This  can  be  checked 
inside  the  while  loop  by  taking  the  convex  hull  of  currentBox  before  and  after  the 
advanceTime  call,  and  passing  that  to  a  function  which  ensures  the  hull  does  not  con¬ 
tain  a  state  which  violates  the  system  constraints.  For  checking  condition  (3),  the  final 
currentBox  value  can  be  used.  Furthermore,  these  checks  can  be  done  at  each  itera¬ 
tion  of  the  refinement;  if  a  reachTimeStep  is  found  such  that  the  three  conditions  of 
the  theorem  are  satisfied,  no  further  refinement  is  necessary  (and  the  complex/smart 
controller  can  be  used). 

5.  EVALUATION 

We  now  present  an  evaluation  of  the  proposed  methodology.3  The  real-time  reachabil¬ 
ity  approach  computes  the  set  of  reachable  states  for  the  safety  controller  as  depicted 
in  the  automaton  representing  the  Simplex  architecture  in  Figure  5.  We  demonstrate 
the  method  through  two  related  case  studies:  a  nonlinear  inverted  pendulum  and  a 
linear  inverted  pendulum.  As  another  benefit  of  the  real-time  reachability  method  de¬ 
scribed  in  this  paper,  it  can  also  work  even  if  the  LMI  approach  cannot  be  used.  We 
note  that  the  LMI  approach  in  general  cannot  be  used  for  nonlinear  systems,  so  its 
application  is  limited.  In  order  to  directly  show  the  advantage  of  the  approach  in  the 
linear  case,  we  use  the  same  case  study  that  demonstrated  the  earlier,  LMI-based 
Simplex  work  [Seto  and  Sha  1999].  The  linear  inverted  pendulum  model  is  obtained 
by  linearizing  the  nonlinear  inverted  pendulum  model,  and  overall,  their  results  are 
comparable  and  were  used  to  validate  against  one  another.  Both  models  are  briefly 
discussed  here,  with  more  details  on  the  nonlinear  and  linear  models  in  the  earlier  re¬ 
port  [Seto  and  Sha  1999].  The  system  is  an  inverted  pendulum  with  state  constraints 
and  input  saturation.  The  physical  system  is  shown  in  Figure  6  and  consists  of  a  DC- 
motor  driven  cart  that  moves  along  a  1-d  track  with  a  pendulum  arm  attached  by  an 


3  As  it  is  difficult  to  present  all  the  details  necessary  to  replicate  our  results  in  the  form  of  a  paper,  the  source 
code  implementation  of  the  real-time  reachability  algorithm  and  our  models  are  available  as  supplementary 
material  and  online  at:  http://www.verivital.com/rtreach/. 
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sw_s 


c  :=  0 

Fig.  5:  Hybrid  automaton  for  the  Simplex  architecture  on  which  the  real-time  reacha¬ 
bility  computation  is  performed  for  the  safety  controller  mode,  where  a  formal  model 
is  available.  Here  A  is  a  control  period,  c  is  a  timer,  Gs  is  a  guard  governing  the  transi¬ 
tion  from  the  safety  to  complex  controllers,  Gc  is  a  guard  governing  the  transition  from 
the  complex  to  safety  controllers,  and  Fs  and  Fc  respectively  denote  the  dynamics  of 
the  overall  closed-loop  system  when  using  the  safety  and  complex  controllers. 


Fig.  6:  An  inverted  pendulum  system  keeps  a  rod  upright  at  an  unstable  equilibrium 
point  by  controlling  a  cart  at  its  base. 


angular  joint  to  the  cart.  The  control  objective  is  to  keep  the  angle  9  of  the  pendulum 
arm  at  0°  measured  from  the  vertical  (i.e.,  to  keep  the  arm  upright). 

There  are  four  state  variables:  cart  position  p,  cart  velocity  v  =  p,  pendulum  arm 
angle  9,  and  pendulum  arm  angular  velocity  uj  =  6.  We  denote  x  as  the  state  vector 
and  p  as  the  position,  seen  together  next  in  Equation  1: 


V 

V 

P 

V 

V 

0 

= 

V 

9 

,  yielding  the  dynamics  x  = 

P 

9 

= 

V 

0 

UJ 

9 

6 

UJ 

(1) 


The  system  is  subject  to  physical  constraints.  The  range  of  p  is  between  [—1,1]  me¬ 
ters,  p  is  between]— 1.0, 1.0]  meters/second,  0  is  between  [—15,15]°,  and  9  is  uncon¬ 
strained  although  the  constraints  on  p  do  impose  limits  on  if).  We  ignore  static  friction 
(with  respect  to  the  cart  wheels  and  ground,  and  with  respect  to  the  pendulum  arm  and 
joint)  and  take  the  armature  inductance  (/.„  =  18  millihenries)  to  be  0  henries  hence 
reducing  the  order  of  the  system  by  making  the  armature  current  state  variable  Ia  a 
function  of  only  Va.  Without  this  simplification,  two  control  states  would  be  necessary. 
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Fig.  7:  Overapproximation  of  the  set  of  reachable  states  computed  by  the  real-time 
reachability  method  for  the  nonlinear  inverted  pendulum  model  Equation  2  and  Equa¬ 
tion  3  for  different  amounts  of  computation  runtime,  2  ms  in  (a),  6  ms  in  (b),  and  20 
ms  in  (c). 


5.1.  Nonlinear  Inverted  Pendulum 

The  inverted  pendulum’s  state  evolves  according  to  a  nonlinear  differential  equation 
x  =  f(x,u).  Specifically, 


f(x,u)  = 


Here,  Di  =  4 M  —  3m,  B  = 


p 

V 

p 

m  (C\+fc)  —  ^lm  cos  (9)  ( C2+fP ) 

_ 

D 

9 

UJ 

0 

M(C2+fP)-^lm  cos  (9)  (Ci  +  /c) 

L  D  J 

(2) 


KaBr, 


K2gKiKb 
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i  ^2  ?  j  c 


Bxv  +  Axe~c*W  sign  (u),  fp  =  Bgu  +  Ag  e~c°  M  sign  (w) ,D  =  l2m  (f  +  f  +  %^) 


l  mz  cos  (Oy 


,C\  =  v 


Ka 


Kb  Kg  K, 


) 


l  m  coz  sin  ( 6 ) 


,  and  C2  =  — 


glm  sin  ( 6 ) 


.  The  pen- 


4  ,  ^  r2  -r  Rar2  j  2  ,““^2-  2 

dulum  model  involves  the  following  parameters:  g  is  gravity,  Ra  is  the  armature  resis¬ 
tance,  r  is  the  driving  wheel  radius,  Jm  is  the  motor  rotor  inertia,  Bm  is  the  motor’s 
coefficient  of  viscous  friction,  Be  is  the  pendulum  joint’s  coefficient  of  viscous  friction, 
Ki  is  the  motor  torque  constant,  Kb  is  the  motor  back-e.m.f.  constant,  Kg  is  the  gear 
ratio,  M  is  the  cart  mass,  m  is  the  pendulum  arm  mass,  l  is  the  pendulum  arm  length, 
fc  is  the  static  friction  force,  and  fp  is  the  viscous  friction  force.  After  evaluating  values 
for  constant  parameters  (the  same  as  those  used  in  [Seto  and  Sha  1999]),  we  have: 


f(x,u)  = 


p 

p 

0 

9 

0.020833w2  sin(0)  — 0.059221u+0.25  cos(0)(O.OOOlw+2.45  sin(0)) 
0.0625  cos(0)2— 0. 604167 


0.000725^+17.7625  sin(0)  — 0. 25  cos(0)(— 0. 25  sin(0)aj2+O.71O657«) 
0.0625  cos(0)2-0. 604167 


(3) 


As  illustrated  in  Figure  7  by  the  decreasing  size  of  the  overapproximation  of  the  set 
of  reachable  states,  the  more  runtime  given  to  the  real-time  reachability  algorithm, 
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the  more  accurate  the  result  (see  also  Figure  8  and  Tables  I,  II,  and  III).  These  results 
illustrate  that  the  real-time  reachability  algorithm  presented  in  this  paper  is  effective 
even  for  hybrid  systems  with  nonlinear  differential  equations.  Thus,  the  results  are 
widely  applicable  to  many  realistic  systems. 

5.2.  Linearized  Inverted  Pendulum 

As  discussed  in  Section  5.1,  the  system  is  in  general  nonlinear,  x  =  f(x,u),  but  to 
apply  the  LMI-Simplex  approach  as  a  part  of  the  unified  Simplex  method  described 
in  this  paper,  we  next  work  with  a  model  linearized  around  the  origin,  which  is  the 
equilibrium  point: 


x  =  Ax  +  Bu. 

The  linearization  is  justified  since  the  control  objective  is  to  stabilize  the  system  in  a 
neighborhood  of  the  vertical  equilibrium,  defined  in  this  coordinate  system  as  0  =  0°, 
which  is  at  the  origin. 

The  plant  system  matrix  and  input  vector  used  in  Equation  5.2  are: 
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where 
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and  64  =  for  all  the  parameters  defined  in  Section  5.1.  Using  the  parameters  from 

the  earlier  Simplex  report  [Seto  and  Sha  1999],  the  A  and  B  matrices  used  in  Equa¬ 
tion  5.2corresponding  to  Equation  4  are: 
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and  B  = 
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-4.44 

The  system  is  stabilized  by  linear  state  feedback  of  the  form  x  =  {A  +  BK )  x.  The 
control  input,  u  =  Kx  is  the  armature  voltage  of  a  DC-motor  (V„)  and  is  constrained 
between  [—4.95,4.95]  volts.  Additionally,  this  control  saturation  prevents  the  system 
from  being  globally  stable.  The  safety  controller  is  designed  following  the  LMI-based 
Simplex  approach  described  in  Section  2.  The  LMI  approach  outputs  a  set  of  gains  for 
the  safety  control  K,  such  that  when  the  input  u  =  Kx  is  used,  the  system  will  remain 
inside  the  ellipsoid  also  output  by  the  method.  Without  saturation,  the  system  evolves 
according  to  x  =  (A  +  BK)  x. 

The  solution  to  this  is  x(t)  =  e(-A+BK^tx0  ,  where  x0  G  R4xl  is  an  initial  condition 
vector.  Note  that,  as  only  0  and  p  are  observable  (in  the  control  theoretic  sense,  but  that 
is,  are  measured  by  sensors),  8  and  x  are  constructed  by  the  first-order  approximations 
8(t)  =  [e(0~^r~mT3)1  and  p(t)  =  ^tKp(^mTA] ,  where  m  is  an  integer  greater  than  one 
(chosen  as  2  by  experimentation).  In  the  safety  and  experimental  controllers,  this  first- 
order  approximation  is  accomplished  by  storing  a  buffer  of  previous  sampled  values. 
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Fig.  8:  Overapproximation  of  the  set  of  reachable  states  computed  by  the  real-time 
reachability  method  for  the  linearized  inverted  pendulum  model  Equation  4  and  Equa¬ 
tion  5  for  different  amounts  of  computation  runtime,  3  ms  in  (a),  7  ms  in  (b),  and  16 
ms  in  (c).  The  plots  illustrate  2-d  projections  of  the  reachable  sets  for  the  linearized 
inverted  pendulum  from  the  state  x  =  [—0.1,  0.85,  0,  0]T  for  reach-time  0.73.  Here, 
the  initial  state  is  outside  of  the  LMI-recoverable  ellipsoid  (xTPx  =  1.56),  but  can 
be  proven  to  reenter  the  ellipsoid  after  0.73  reach-time,  despite  the  presence  of  input 
saturation. 


5.3.  Feasible  and  Stabilizable  Regions 

Next,  we  discuss  how  to  compute  the  feasible  and  stabilizable  regions,  defined  pre¬ 
viously  in  Section  2.1.  We  use  YALMIP  [Lofberg  2004],  the  SDPT-3  [Toh  et  al.  1999] 
solver,  and  Matlab  to  solve  the  following  semidefinite  quadratic  programming  problem 
and  under-approximate  the  recoverable  states  for  the  safety  controller.  For  computing 
the  stabilizable  region  for  the  safety  controller,  we  find  the  gain  vector  during  the  opti¬ 
mization.  The  problem  is  to  maximize  the  volume  of  the  ellipsoid  (and  thus  maximize 
the  set  of  recoverable  states)  defined  by: 

TZ  =  {x  \  xTPx  <  1}.  (6) 

The  LMI  to  find  the  positive  definite  P  may  formulated  as: 

min  logdetQ-1 

subject  to  QAt  +  AT  Q  <0,  Q  >  0,  a^Qcik  <1,  k  =  1, . . . , n, 

where  A  =  A  +  BK,  Q  =  P_1,  and  the  au  for  k  £  {1, ...,  n}  encode  the  state  and  control 
constraints.  Full  details  of  this  process  are  given  in  Appendix  A2  of  the  LMI-Simplex 
technical  report  [Seto  and  Sha  1999]. 

Variants  of  this  process  may  either  take  a  given  gain  vector  K  or  find  a  gain  vector 
K  [Seto  and  Sha  1999].  For  our  use,  the  output  of  this  process  is  both  the  gain  vector  K 
and  the  matrix  P  defining  a  subset  of  the  recoverable  states  TZ,  such  that  when  the  gain 
matrix  is  used  for  the  safety  controller,  and  the  state  is  in  TZ,  the  state  is  guaranteed 
to  stay  in  1Z  indefinitely  (since  V (x)  =  xTPx  is  a  Lyapunov  function).  Furthermore,  all 
the  constraints  (including  saturation  limits)  are  satisfied  for  all  states  in  TZ.  The  gain 
vector  K  produced  for  the  described  pendulum  system  is  [0.4072,  7.2373, 18.6269, 3.6725]. 
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5.4.  Real-Time  Reachability  Design 

In  order  to  make  use  of  the  real-time  reachability  algorithm  described  in  Section  4, 
the  user  must  provide  a  function  that  minimizes  /  maximizes  the  derivative  in  an 
arbitrary  box  of  the  state  space.  The  model  used  of  the  system  is  the  version  described 
above,  linearized  about  the  origin.  Thus,  the  system  dynamics  are  x  =  Ax  +  Bu.  When 
computing  reachability  for  the  safety  controller,  the  gain  vector  K  computed  using  the 
LMI  approach  is  used,  and  u  =  Kx.  However,  due  to  saturation,  u  is  limited  to  be  in 
the  range  [—4.95,4.95]  volts. 

An  alternative  unified  design  could  make  use  of  the  nonlinear  pendulum  model 
from  Section  5.1,  since  the  described  reachability  algorithm  is  not  limited  to  linear 
systems.  An  advantage  of  such  a  design  would  be  that  it  would  permit  the  system 
state  to  go  outside  of  the  linearization  region  (in  our  formulation  with  the  LMI,  the 
recoverable  region  of  Definition  2  specified  in  Lemma  5).  It  would  be  interesting  as 
a  future  investigation  to  see  how  much  more  could  be  gained  by  allowing  states  out¬ 
side  of  the  linearization  region,  although  such  a  gain  probably  strongly  depends  on 
the  system  being  analyzed  and  the  size  of  the  linearization  region.  For  our  purposes, 
Lyapunov’s  indirect  method  ensures  all  states  within  the  LMI  ellipsoids  are  locally 
asymptotically  stable.  We  recall  that  roughly  Lyapunov’s  indirect  method  states  that 
a  nonlinear  system  is  locally  (in  a  neighborhood  of  the  equilibrium  point)  asymptot¬ 
ically  stable  if  its  lineraization  about  an  equilibrium  point  is  globally  asymptotically 
stable  [Khalil  2002].  The  bound  specified  in  the  proof  of  Lyapunov’s  indirect  method 
gives  a  conservative  underapproximation  of  the  linearization  region  (what  is  typically 
called  the  domain  of  attraction).  More  sophisticated  piecewise  linear  Lyapunov  func¬ 
tions  would  yield  less  conservative  estimates  of  domain  of  attraction. 

For  linear  systems,  the  minimum  and  maximum  derivative  for  any  box  in  the  state 
space  occurs  at  a  corner  of  the  box.  Thus,  it  is  sufficient  to  sample  all  the  corners  and 
take  the  minimum  and  maximum  due  to  convexity  and  existence  of  optima  of  convex 
(here,  linear)  functions  over  convex  sets.  This  will  scale  exponentially  with  the  number 
of  dimensions  (in  the  four-dimensional  model  here,  there  are  16  corners  to  sample),  so 
for  larger-dimension  systems  it  may  become  necessary  to  examine  the  signs  of  the 
linear  matrix  in  order  to  pick  out  the  min/max  corner  more  efficiently.  One  additional 
complication  of  the  linearized  model  is  the  presence  of  saturation.  This  is  handled  by 
computing  the  input  u  at  each  corner,  and  capping  it  at  the  saturation  limits  before 
computing  x  =  Ax  +  Bu.  To  summarize,  for  each  corner  of  the  passed-in  box,  x  is 
computed,  and  then  the  minimum  or  maximum  is  taken  over  all  the  corners.  The  C 
language  program  that  computes  x  for  a  given  dimension,  given  a  point  (corner  of  the 
box),  is  provided  in  Algorithm  2. 

Another  function  that  must  be  provided  by  the  user  is  used  to  determine  whether  a 
given  box  is  contained  entirely  inside  the  recoverable  region  1Z.  This  is  used  to  check 
whether  the  final  state  (box)  is  guaranteed  to  be  recoverable.  To  do  this  for  a  single 
point,  it  suffices  to  know  the  current  state  x  and  the  potential  matrix  P  that  defines 
the  recoverable  ellipsoid  (output  by  the  LMI  optimization),  and  checking  xTPx  <  1.  To 
check  this  condition  for  a  box,  we  check  every  corner  point  of  the  box. 

One  further  function  provided  by  the  user  checks  is,  during  computation,  whether 
the  reachable  region  contains  an  unsafe  state.  In  our  case,  this  is  a  state  that  is  outside 
of  the  linearization  region  where  the  model  is  considered  valid.  Since  the  constraints 
are  all  linear,  it  suffices  to  check  if  all  of  the  corners  of  each  box  are  in  the  lineariza¬ 
tion  region.  This  computation  is  done  at  runtime  to  prevent  saving  the  reachable  set 
of  states.  The  box  passed  in  to  this  function  consists  of  the  bounding  box  of  subse¬ 
quent  steps  of  the  real-time  reachability  algorithm,  which  represents  the  sets  of  states 
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Algorithm  2  This  function  returns  the  derivative  at  a  given  point.  The  min/max 
derivative  function  would  compute  the  derivative  at  each  corner  of  a  passed-in  box, 
and  take  the  minimum/maximum.  The  inputs  are  a  particular  dimension  dim,  the 
number  of  total  dimensions  n,  a  pointer  to  a  state  vector  state,  which  is  an  array 
of  length  n,  and  a  control  saturation  u_sat.  For  the  pendulum  case  study,  n  =  4  and 
u sat  =  4.95. 


1 

double  derivative_at_point  (  int  dim,  int  n, 

double  *  state  ,  double  u_sat)  { 

double  rv  =  0; 

3 

double  u  =  0; 

5 

//  calculate  the  A  *  x  part 

for  (  int  i  =  0;  i  <  n;  ++i) 

7 

rv  +=A[dim][i]  *  state  [i]; 

9 

//  calculate  the  B*u  part  ,  starting 
for  (  int  i  =  0;  i  <  n;  ++i) 

with  u  =  K  *  x 

11 

u  +=  K[  i ]  *  state [ i  ] ; 

13 

//  account  for  input  saturation 

if  (u  <  — u_sat)  u  =  — u_sat ; 

15 

else  if  (u  >  u_sat)  u  =  u_sat ; 

17 

//  B  *  u 

rv  +=  B[dim]  *  u; 

19 

return  rv ; 

} 

reachable  between  two  time  steps,  say  U  and  ti+6,  where  5  is  the  advanceReachTime 
in  Algorithm  1. 

5.5.  Comparison  between  Simplex  with  LMI  and  Real-Time  Reachability 

We  now  provide  a  comparison  between  control  based  on  the  TZ  from  the  LMI  approach 
above,  and  the  switching  condition  produced  by  the  proposed  unified  approach  that 
uses  real-time  reachability.  For  real-time  reachability,  we  implemented  the  algorithm 
from  Section  4.  In  order  to  be  usable  in  a  real-time  control  system,  our  implementa¬ 
tion  was  written  in  C  and  had  no  dynamic  memory  allocations  or  recursion,  and  used 
no  nonstandard  external  libraries.  In  our  implementation,  we  would  call  the  real-time 
reachability  C  code  from  within  Matlab  on  either  Linux  and  Windows.  For  the  ex¬ 
periments  described  here,  we  first  used  a  modern  laptop  with  a  quad-core  Intel  Core 
i7-2800MQ  processor  and  32GB  RAM  (although  the  computation  does  not  require  sig¬ 
nificant  memory  as  described  earlier).  Next,  we  additionally  evaluated  the  methods 
on  embedded  platforms.  The  first  embedded  platform  is  a  BeagleBone  Black  devel¬ 
opment  board  with  a  1GHz  ARM  processor  and  512MB  RAM  running  Debian  Linux 
with  the  Xenomai  real-time  Linux  extensions.  The  second  embedded  platform  is  an 
Arduino  Yun,  which  has  both  a  400MHz  MIPS  processor  and  a  16MHz  8-bit  Atmel 
AVR  ATmega32u4  processor,  and  we  used  the  ATmega32u4  for  our  evaluation,  in  part 
to  validate  our  claims  on  minimal  resources  requirements.  Together,  these  evaluations 
validate  our  claims  that  the  real-time  reachability  method  is  cross-platform  and  re¬ 
quires  minimal  processing  resources.  The  effort  to  port  from  the  original  x86  imple¬ 
mentation  [Bak  et  al.  2014]  to  both  the  ARM  and  AVR  implementations  took  about 
two  weeks  of  development  time,  which  from  a  systems  development  standpoint  is  min¬ 
imal  given  the  insurmountable  difficulties  that  would  exist  in  porting  all  the  libraries 
required  in  other  existing  hybrid  systems  reachability  approaches. 
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Fig.  9:  Estimated  projections  are 
shown  of  the  LMI-Simplex  recoverable 
region  TZ  (cyan  center  set),  real-time 
reachability  recoverable  region  (green 
middle  set),  Simulink/Stateflow  sim¬ 
ulations  that  converge  (yellow  mid¬ 
dle  set),  and  simulations  that  diverge 
(red  exterior  set)  where  6  =  0.19  rad 
(~10.89°)  and  8  =  0.18  rad  (~10.31°)  per 
second. 
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Fig.  10:  Estimation  of  LMI-Simplex  re¬ 
coverable  region  TZ  (cyan  center  set), 
real-time  reachability  recoverable  re¬ 
gion  (green  interior  set),  Simulink/S¬ 
tateflow  simulations  that  converge 
(yellow  middle  set),  and  simulations 
that  diverge  (red  exterior  set)  shown 
on  the  projection  of  8  and  9  =  u  onto 
the p  =  0m  and  v  =  0  m/s  plane. 


One  remaining  input  for  the  algorithm  is  the  reach-time  necessary  for  a  specific  state 
to  reenter  7 Z  (the  time  S+a  from  Theorem  6).  This  was  approximated  using  Euler-based 
simulation,  which  added  a  fixed  overhead  at  the  start  of  the  computation.  For  states 
slightly  outside  of  TZ,  the  necessary  reach-time  was  typically  in  the  hundreds  of  mil¬ 
liseconds.  Since  the  reachability  computation  incurs  error  due  to  over  approximation, 
we  compute  the  set  of  reachable  states  for  slightly  more  (1.2  times)  than  the  time  the 
simulation  took  to  reach  TZ.  If  the  Euler  simulation  did  not  enter  TZ  by  some  upper 
bound  (4  seconds  reach-time),  the  state  was  considered  unrecoverable.  A  projection  of 
the  computed  overapproximation  of  the  set  of  reachable  states  for  various  runtimes  is 
shown  in  Figure  8.  As  more  computation  runtime  is  added,  the  accuracy  increases,  as 
indicated  by  the  size  of  the  set  decreasing. 

One  difference  between  the  approaches  is  that  the  LMI-Simplex  method  needs  to 
reason  about  one-step  reachability  of  the  plant  state  for  any  complex/smart  controller 
command  in  order  to  compute  the  distance  d  in  Figure  2.  The  proposed  online  ap¬ 
proach,  in  contrast,  knows  what  complex-controller  command  will  be  applied  and  can 
use  that  as  part  of  the  reachability  computation.  For  this  reason,  we  restrict  the  com¬ 
parison  to  only  examine  the  recoverable  region  for  the  safety  controller.  In  this  way, 
we  do  not  give  our  approach  the  advantage  of  knowing  exactly  what  command  the 
complex/smart  controller  is  using. 

Our  comparison  shows  three  different  approaches  for  estimating  the  recoverable 
region  (Figures  9  and  10).  First,  using  the  LMI-only  Simplex  we  get  a  subset  of  the 
recoverable  region  TZ.  Next,  using  a  simulation-based  analysis  in  Matlab,  we  can  see 
an  approximation  of  all  recoverable  states,  which  would  be  an  ideal  switching  set.  If 
the  simulation  returns  to  a  steady  state  then  the  initial  point  is  marked  as  existing  in 
the  recoverable  set.  Finally,  we  show  the  states  that  the  real-time  reachability-based 
approach  can  guarantee  as  recoverable,  which  is  somewhere  between  the  previous  two 
regions.  For  these  experiments,  in  order  to  be  runnable  in  the  control  loop,  the  runtime 
for  the  reachability  code  was  capped  at  20  ms. 
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The  stabilizable  regions  for  p  and  p  of  the  controller  is  seen  in  Figure  9  and  the  re¬ 
gions  for  6  and  6  of  each  controller  are  in  Figure  10.  One  reason  why  the  runtime  reach¬ 
ability  approach  can  recover  more  states  is  that  the  recoverable  set  contains  states 
where  input  saturation  occurs,  whereas  the  set  TL  contains  no  such  states.  The  largest 
improvements  in  the  switching  set  for  the  real-time  approach  occur  under  this  satu¬ 
ration  situation,  because  reachability  is  able  to  reason  about  the  behavior  of  the  satu¬ 
rated  system.  Another  reason  for  the  improvement  is  that  the  LMI-produced  switching 
set  must  be  an  ellipsoid,  whereas  the  true  set  of  recoverable  states  can  be  an  arbitrary 
(even  non-convex)  shape.  This  is  seen  in  Figure  9,  where,  since  the  projection  is  near 
the  maximum  values  of  6  and  0,  the  LMI  ellipsoid  projected  onto  this  plane  is  small. 
In  Figure  10  the  LMI-Simplex  recoverable  region  is  clearly  ellipsoidal  (as  expected 
from  Equation  6).  In  both  Figures  9  and  10,  the  benefit  of  using  real-time  reachability 
is  highlighted  by  the  larger  provably  safe  recoverable  region.  In  both  cases,  even  for  a 
20  ms  runtime,  the  set  of  states  proven  recoverable  using  real-time  reachability  is  very 
close  to  the  simulations  that  converge,  which  means  that  the  real-time  reachability  is 
close  to  optimal  in  estimating  the  actual  recoverable  region. 

Next,  we  evaluated  the  effect  of  varying  the  runtime  in  real-time  reachability 
method  on  the  resultant  switching  set,  which  is  summarized  in  Table  I.  For  this  ta¬ 
ble,  we  sampled  the  state-space  uniformly  between  the  state  bounds  presented  ear¬ 
lier  using  15  points  in  each  dimension  (so  154  =  50625  points)  in  the  hyper-rectangle 
—  1.25  <  p  <  1.25  (m),  —1.2  <  p  <  1.2  (m/s),  —20  <  9  <  20  (degrees),  and  —30  <  9  <  30 
(degrees/s).  The  columns  LMI,  Real-Time,  Sim,  and  Unrecov  list  the  number  of  recov¬ 
erable  points  for  each  approach  (in  terms  of  recoverable  states,  notice  that  LMI  C  Re- 
alTime  C  Sim),  as  measured  by  the  uniform  sampling.  The  column  Recoverable  is  the 
comparison  of  the  number  of  states  verified  safe  in  the  proposed  unified  method  with 
real-time  reachability  over  the  earlier  LMI-Simplex  approach.  The  improvement  is  an 
estimate  of  the  increased  state-space  size  (volume)  allowed  using  our  real-time  reach¬ 
ability  method,  over  using  only  the  LMI-based  recoverable  region.  Since  the  real-time 
recoverable  states  contain  all  the  LMI-Simplex  states,  the  improvement  is  calculated 
as:  (#RealTime  Points  +  #LMI  Points)/(#LMI  Points).  For  a  runtime  of  20  ms,  the  im¬ 
provement  in  volume  of  the  switching  set  is  estimated  at  227%,  whereas  based  on  sim¬ 
ulations  we  estimate  the  maximum  possible  improvement  in  Recoverable  to  be  around 
247%  (calculated  as  (#Sim  Points  +  ^RealTime  Points  +  #LMI  Points)/(#LMI  Points)). 

We  experimented  with  increasing  the  number  of  samples  up  to  30  points  in  each  di¬ 
mension,  which  yielded  similar  improvements,  and  in  the  limit  as  the  number  of  sam¬ 
ples  tends  to  infinity,  we  would  converge  to  the  exact  improvement.  However,  these 
approximations  are  reasonable  based  on  the  consistency  of  our  experimental  results 
(e.g.,  20  ms  runtime  for  15  samples  is  about  a  227%  improvement,  and  it  is  also  about  a 
230%  improvement  for  30  samples).  As  expected,  as  the  runtime  allowed  for  real-time 
reachability  increases,  the  improvement  increases  since  the  real-time  reachability  im¬ 
plementation  uses  an  anytime  approach  and  refines  the  precision  of  the  reachability 
computation  based  on  available  runtime.  Even  for  small  runtimes  (e.g.,  5ms),  the  im¬ 
provement  is  already  significant  at  over  200%  more  provably  recoverable  states,  which 
makes  the  approach  promising  for  implementation  in  real-time  control  loops. 

5.6.  Comparison  on  ARM  and  Arduino  AVR  ATmega32u4  Embedded  Hardware  Platforms 

Next,  we  compare  the  benefit  of  using  our  real-time  reachability  approach  versus  the 
LMI-Simplex  method  on  actual  embedded  hardware  platforms.  The  first  hardware 
platform  is  an  ARM  processor  in  the  TI  Sitara  system-on-chip  used  in  the  CircuitCo 
BeagleBone  Black  development  kit.  The  specific  ARM  processor  is  an  AM335x  1GHz 
ARM  Cortex-A8  with  the  NEON  floating-point  accelerator  and  access  to  512  MB  DDR3 
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Table  I.  Intel  Core  i7  x86-64:  PC  evaluation  summary  of  experiments  varying  run¬ 
time. 


Runtime  (ms) 

LMI 

RealTime 

Sim 

Unrecov 

Recoverable 

5 

5473 

5971 

14323 

24858 

209% 

20 

5473 

6753 

13541 

24858 

223% 

40 

5473 

6974 

13320 

24858 

227% 

50 

5473 

7081 

13213 

24858 

229% 

75 

5473 

7109 

13185 

24858 

230% 

100 

5473 

7183 

13111 

24858 

231% 

200 

5473 

7273 

13021 

24858 

233% 

500 

5473 

7338 

12956 

24858 

234% 

1000 

5473 

7382 

12912 

24858 

235% 

2000 

5473 

7424 

12870 

24858 

236% 

3000 

5473 

7428 

12866 

24858 

236% 

4500 

5473 

7448 

12846 

24858 

236% 

6000 

5473 

7455 

12839 

24858 

236% 

RAM.  The  experiments  were  conducted  on  a  Debian  Linux  distribution  with  a  kernel 
modification  to  use  the  Xenomai  real-time  Linux  extensions,  enabling  use  of  real-time 
operating  system  (RTOS)  features  within  Linux.  A  summary  of  experimental  results 
are  reported  in  Table  II.  Here  we  can  see  that  for  reasonable  runtimes  even  on  an 
embedded  platform  (tens  of  milliseconds),  the  approach  presented  in  this  paper  has 
an  improvement  of  around  1.5  to  2  times  over  the  LMI  approach.  For  runtimes  on 
the  order  of  hundreds  of  milliseconds  to  seconds,  the  approach  yields  similar  improve¬ 
ments  to  the  desktop  implementation.  For  this  table  (as  with  Table  I),  we  sampled  the 
state-space  uniformly  between  the  state  bounds  presented  earlier  using  15  points  in 
each  dimension  (so  154  =  50625  points)  in  the  same  hyper-rectangle  used  in  the  earlier 
experiment,  specifically  —1.25  <  p  <  1.25  (m),  —1.2  <  p  <  1.2  (m/s),  —20  <  9  <  20 
(degrees),  and  -30  <  9  <  30  (degrees/s). 

The  second  hardware  platform  is  the  Arduino  Yun.  The  Yun  has  both  a  400  MHz 
MIPS  processor  and  a  16  MHz  8-bit  Atmel  AVR  ATmega32u4.  For  this  evaluation, 
we  use  the  16  MHz  ATmega32u4  processor,  which  is  representative  of  small,  mem¬ 
ory  constrained  embedded  devices.  The  ATmega32u4  has  available  2.5  KB  SRAM,  32 
KB  of  flash  memory,  but  because  the  real-time  reachability  method  does  not  use  any 
dynamic  memory  allocation  and  does  not  rely  on  any  non-standard  libraries,  we  are 
able  to  run  it  on  the  platform  in  spite  of  the  processing  and  memory  constraints.  Al¬ 
though  the  implementation  runs  with  the  restricted  resources,  the  runtime  is  notice¬ 
ably  higher  than  on  the  with  other  processors.  In  this  case,  the  system  would  only 
stand  to  benefit  if  the  dynamics  were  sufficiently  slow  (so  a  runtime  of  seconds  would 
be  tolerable),  or  if  we  further  optimized  parts  of  the  implementation  for  the  limited  re¬ 
sources  (changing  software  floating-point  computations  to  use  fixed-point,  since  there 
is  no  FPU  on  the  ATmega32u4.  A  summary  of  experimental  results  for  the  AVR  are 
reported  in  Table  III.  For  this  table  (unlike  in  Tables  I  and  II),  we  sampled  the  state- 
space  uniformly  between  the  state  bounds  presented  earlier  using  12  points  in  each 
dimension  (so  124  =  20736  points)  in  the  same  hyper-rectangle  as  the  earlier  experi- 
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Table  II.  BeagleBone  Black  ARM:  Embedded  system  evaluation  summary  of  experi¬ 
ments  varying  runtime. 


Runtime  (ms) 

LMI 

RealTime 

Sim 

Unrecov 

Recoverable 

5 

5473 

2270 

18024 

24858 

141% 

20 

5473 

3832 

16462 

24858 

170% 

40 

5473 

4613 

15681 

24858 

184% 

50 

5473 

4617 

15677 

24858 

184% 

75 

5473 

5350 

14944 

24858 

198% 

100 

5473 

5361 

14933 

24858 

199% 

200 

5473 

5968 

14326 

24858 

209% 

500 

5473 

6721 

13573 

24858 

223% 

1000 

5473 

6952 

13342 

24858 

227% 

2000 

5473 

7107 

13187 

24858 

230% 

3000 

5473 

7110 

13184 

24858 

230% 

4500 

5473 

7216 

13078 

24858 

232% 

6000 

5473 

7216 

13078 

24858 

232% 

Table  III.  Arduino  Atmel  AVR  ATmega32u4:  Embedded  system  evaluation  summary 
of  experiments  varying  runtime. 


Runtime  (ms) 

LMI 

RealTime 

Sim 

Unrecov 

Recoverable 

100 

2088 

0 

8226 

10422 

100% 

500 

2088 

192 

8034 

10422 

109% 

1000 

2088 

566 

7660 

10422 

127% 

2000 

2088 

879 

7347 

10422 

142% 

3000 

2088 

882 

7344 

10422 

142% 

4500 

2088 

1198 

7028 

10422 

157% 

ments,  specifically  —1.25  <  p  <  1.25  (m),  —1.2  <  p  <  1.2  (m/s),  —20  <  9  <  20  (degrees), 
and  —30  <  9  <  30  (degrees/s).  While  the  AVR  is  too  resource  constrained  to  be  able 
to  improve  the  states  usable  in  the  control  period  time  (of  20  ms)  and  requires  on  the 
order  of  hundreds  of  milliseconds  to  seconds  to  yield  an  improvement,  this  is  to  the 
best  of  our  knowledge,  the  first  demonstration  of  a  reachability  method  in  a  resource 
constrained  embedded  system  of  this  scale.  We  also  highlight  that  simply  performing 
a  simulation  on  the  AVR  requires  about  hundreds  of  milliseconds  of  runtime. 

6.  RELATED  WORK 

The  Simplex  Architecture  [Sha  2001;  Seto  and  Sha  1999]  has  been  used  extensively 
to  provide  guarantees  for  systems  that  use  untrusted  logic.  It  has  been  used  for  sys¬ 
tems  ranging  from  off-road  vehicles  [Bak  2009],  to  models  of  airplanes  [Seto  et  al. 
1999],  to  fleets  of  remotely  controlled  cars  [Crenshaw  et  al.  2007],  to  networked  con¬ 
trol  systems  [Yao  et  al.  2013].  Recently,  variants  of  Simplex  have  been  proposed  to 
account  for  physical-system  (plant)  failures  [Wang  et  al.  2013],  faults  in  the  OS  or 
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microprocessor  [Bak  et  al.  2009],  and  to  check  for  security  intrusions  [Mohan  et  al. 
2013].  Simplex  is  closely  related  to  Run-Time  Assurance  (RTA)  methods  [Clark  et  al. 
2013;  Murthy  2012].  RTA  methods  were  used  to  construct  a  safe  supervisory  control 
system  for  a  simulation  of  a  high-altitude  unmanned  aerial  vehicle  [Aiello  et  al.  2010]. 
Here,  a  transition  function  that  projects  the  current  state  to  a  future  state  was  used 
to  determine  the  switching  boundary.  This  transition  function  as  well  as  the  recov¬ 
erable  states  were  determined  through  extensive  simulation  and  online  prediction  of 
trajectories.  The  proposed  real-time  reachability  approach  in  this  work  could  be  used 
to  provide  verified  bounds  on  the  transition  function  used  in  RTA  methods.  This  work 
also  mentions  the  interesting  idea  of  using  an  online/offline  design  for  switching  mod¬ 
ule  logic  by  leveraging  a  simplified  model  of  the  plant  dynamics,  and  taking  the  model 
error  into  account  when  doing  the  switching,  which  could  reduce  the  complexity  of  the 
online  reachability  computation. 

Earlier  work  has  also  integrated  traditionally  non-real-time  search  approaches 
within  real-time  systems  [Musliner  and  Durfee  1995].  In  this  approach,  Al  planning 
techniques  were  discussed  in  the  context  of  real-time  systems,  and  two  categories  of 
possible  integation  were  proposed:  1.  the  non-real-time  algorithms  were  adapted  to 
run  in  a  real-time  fashion,  or  2.  they  were  run  in  a  supervisory  mode,  not  as  part  of 
the  real-time  control  loop.  Real-time  reachability  would  fall  in  the  former  category  in 
this  classification. 

A  related  notion  to  Simplex  in  control  theory  is  that  of  a  viability  kernel  [Aubin 
1991].  A  viability  kernel  is  a  set  of  states  where  there  exists  a  trajectory  that  stays 
within  a  predefined  environment.  Viability  kernels  can  be  approximated  for  linear  sys¬ 
tems,  for  example,  by  using  analysis  of  random  directions  in  the  state  space  [Gillula 
et  al.  2014].  Reachability  analysis  of  hybrid  systems  has  also  been  extensively  re¬ 
searched  in  the  last  20  years  [Gueguen  et  al.  2009].  Reachability  analysis  tools  exist 
for  classes  of  systems  with  timed  [Bengtsson  et  al.  1996],  rectangular  [Henzinger  et  al. 
1997;  Frehse  2008;  Johnson  and  Mitra  2014],  linear  [Frehse  et  al.  2011;  Frehse  2008], 
and  nonlinear  [Ratschan  and  She  2007;  Tiwari  2008;  Bak  2013a;  Chen  et  al.  2012; 
Benvenuti  et  al.  2014;  Duggirala  et  al.  2015]  dynamics,  with  varying  degrees  of  accu¬ 
racy  and  scalability.  Other  bounded  model  checking  (BMC)  tools  for  hybrid  systems 
built  on  satisfiability  modulo  theories  (SMT)  solvers  also  exist  [Eggers  et  al.  2011;  Gao 
et  al.  2013].  However,  to  the  best  of  our  knowledge,  the  algorithms  in  earlier  reach¬ 
ability  and  BMC  tools  were  all  designed  for  offline  analysis,  and  not  for  real-time, 
in-the-loop  computation.  Specifically,  real-time  reachability  requires  performance  to 
be  predictable,  which  is  difficult  to  ensure  when  there  are  large  external  libraries, 
huge  code  bases,  and  signifanct  use  of  dynamic  memory.  For  example,  one  popular 
reachability  analysis  tool  for  affine  hybrid  automata  is  SpaceEx,  which  requires  at 
least  eight  external  libraries:  Parma  Polyhedra  Library  (PPL)  [Bagnara  et  al.  2008], 
Boost  C++  Libraries,  GNU  Multiple  Precision  Arithmetic  Library  (gmplib),  GNU  Lin¬ 
ear  Programming  Kit  (glpk),  SUNDIALS  (Solver  Suite)  [Hindmarsh  et  al.  2005],  aaflib, 
ublasJama,  and  TinyXML  [Frehse  et  al.  2011]. 4  Another  recent  tool,  C2E2  relies  on 
at  least  eleven  external  libraries:  GNU  Linear  Programming  Kit  (glpk  and  pyglpk), 
GNU  Parser  Generator  (Bison),  Fast  Lexical  Analysis  (FLEX),  Python,  Python  Pars¬ 
ing  Libraries  (Python-PLY),  GTK  Libraries  for  Python  (PyGTK),  Plotting  Libraries  for 
Python  (Matplotlib),  Packing  Configurations  Library  (pkg-config),  GNU  Autoconf  (au- 
toconf),  Python  XML  Library  (lxml),  and  Parma  Polyhedral  Library  (PPL).5  While  sev¬ 
eral  of  these  libraries  would  not  need  to  be  executed  in  the  reachability  computation 
(such  as  those  related  to  parsing  and  package  management),  several  libraries  (PPL, 


4http://spaceex.imag.fr/licensing-45 

5https://publish.illinois.edu/c2e2-tool/download/ 
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gmplib,  glpk,  and  SUNDIALS)  are  fundamental  to  the  reachability  computations.  For 
these  core  libraries,  it  would  be  essentially  impossible  to  convert  SpaceEx  or  C2E2  to  a 
real-time  implementation,  as  several  of  these  libraries  are  incredibly  complex  (specifi¬ 
cally  PPL,  gmplib,  glpk,  and  SUNDIALS). 

The  real-time  reachability  approach  described  in  this  paper  primarily  solves  the 
problem  of  computing  the  continuous  successors  in  a  hybrid  automaton,  although  it  can 
also  be  applied  invariant-disjoint  hybrid  dynamics.  Research  in  computing  continuous 
successors  is  related  to  validated  integration,  which  traditionally  has  been  done  using 
interval  analysis  [Moore  1966],  as  well  as  intervals  with  preconditioning  to  reduce 
wrapping-effect  error  [Stauning  1997].  More  recently,  Taylor  models  have  also  been 
proposed  as  an  alternative  shown  to  provide  superior  long-term  error  control  [Neher 
et  al.  2007],  and  this  is  has  been  integrated  into  a  more  full  hybrid  automaton  model 
checker  [Chen  et  al.  2012].  However,  the  challenge  for  runtime  approaches  such  as 
the  one  proposed  in  this  paper  is  more  with  quick  computation  of  reasonable  accuracy 
rather  than  long-term  error  control,  and  we  are  unaware  of  any  previous  real-time 
validated  integration  approaches. 

Some  recent  work  performs  online  reachability  computation  with  existing,  non-real¬ 
time  algorithms.  This  can  be  used,  for  example,  when  systems  do  not  have  statically- 
known  models  [Bu  et  al.  2011].  This  work,  however,  treats  the  reachability  computa¬ 
tion  as  a  black-box,  which  may  or  may  not  complete  (because  it  does  not  use  a  real-time 
reachability  algorithm).  Another  work  also  uses  existing  reachability  approaches  such 
as  PHAVer  [Frehse  2008]  in  a  medical  safeguard  system  [Li  et  al.  2012],  and  results  in 
a  system  which  may  add  safety,  but  only  if  the  computation  completes  on  time.  While  a 
theoretical  upper  bound  on  execution  time  may  be  formulated  due  to  decidability  of  the 
particular  class  of  hybrid  automata  considered  [Li  et  al.  2014],  the  implementation  of 
PHAVer  does  not  provide  such  guarantees,  and  it  is  not  clear  that  such  a  bound  would 
be  usable  or  too  pessimistic.  A  real-time  reachability  algorithm  that  always  provides 
an  answer  like  our  approach  could  be  integrated  into  both  of  these  approaches. 

Finally,  the  results  of  formal  approaches  are  only  as  good  as  the  model  they  are  pro¬ 
vided.  Accurate  system  identification  [Soderstrom  and  Stoica  1988]  is  therefore  essen¬ 
tial.  The  approach  here  reduces  pessimism  in  the  switching  logic  for  a  given  model.  Ac¬ 
curacy  and  validation  of  the  model  itself  is  an  important  problem,  but  beyond  the  scope 
of  this  work.  Recent  approaches  from  the  hybrid  systems  community,  however,  have  be¬ 
gun  made  use  of  runtime  monitors  to  do  online  checking  of  model  accuracy  [Mitsch  and 
Platzer  2014]. 

7.  CONCLUSION  AND  FUTURE  WORK 

In  this  work,  we  have  proposed  an  alternate  unified  design  for  Simplex  that  leverages 
two  existing  design  methodologies  based  on  control-theoretic  LMI  optimization  and 
hybrid  systems  reachability.  Our  unified  approach  extends  the  region  where  the  com¬ 
plex/smart  controller  enabling  smart  autonomy  can  be  used  by  leveraging  a  real-time 
reachability  computation,  and  thus  decreases  conservatism  in  the  switching  logic.  Us¬ 
ing  a  runtime  of  20ms  (which  matches  the  control  loop  period  time),  we  were  able  to 
expand  the  set  of  states  where  the  complex/smart  controller  could  be  used  by  227%, 
whereas  we  estimated,  through  simulation,  that  the  maximum  improvement  possi¬ 
ble  was  approximately  247%.  Even  with  a  reduced  real-time  reachability  runtime  of 
5ms,  we  were  able  to  improve  upon  the  LMI-based  Simplex  design  by  213%.  On  em¬ 
bedded  processors,  we  were  also  able  to  increase  the  complex/smart  controller  region 
by  a  factor  of  1.5  to  2.0,  although  for  an  8-bit  microcontroller  the  current  implemen¬ 
tation  was  not  fast  enough  for  use  at  the  frequency  of  the  control  loop.  This  improve¬ 
ment  was  demonstrated  in  an  evaluation  that  uses  the  exact  system  previously  used 
to  demonstrate  the  LMI-based  Simplex  design  approach,  an  inverted  pendulum  with 
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input  saturation.  The  real-time  reachability  computation  is  able  to  predict  the  behav¬ 
ior  of  the  system  despite  saturation,  significantly  expanding  the  usable  complex/smart 
controller  region. 

To  the  best  of  the  authors’  knowledge,  this  is  the  first  work  to  present  a  viable 
real-time  reachability  algorithm  based  on  the  real-time  systems  notion  of  imprecise 
computation.  The  algorithm  will  always  return  an  over-approximation  of  the  set  of 
reachable  states,  with  better  accuracy  as  more  computation  time  is  given.  The  key 
difference  between  online  reachability  compared  with  offline  reachability,  besides  con¬ 
strained  runtime  and  resources,  is  that  quick  results  are  preferable  to  long-term  er¬ 
ror  control.  In  our  evaluation,  for  example,  we  were  able  to  demonstrate  significant 
improvement  in  the  complex/smart  controller  region  by  using  tens  of  milliseconds  of 
computation  time  to  bound  the  future  behavior  of  the  system  for  the  next  hundreds  of 
milliseconds.  Together,  our  evaluation  on  actual  embedded  hardware  platforms  includ¬ 
ing  ARM  processors  and  Atmel  AVR  microcontrollers  illustrates  the  embedded  usage 
feasibility  of  using  the  real-time  reachability  method.  Other  reachability  algorithms 
also  contain  parameters  which  could  be  tuned  to  have  some  control  over  the  compu¬ 
tation  time,  such  as  the  sampling  time  used  in  the  Le  Geurnic-Girard  (LGG)  scenario 
in  SpaceEx  [Frehse  et  al.  2011],  and  we  plan  to  investigate  better  approaches  for  real¬ 
time  reachability. 

Real-time  reachability  has  applications  beyond  just  determining  Simplex  switch¬ 
ing  logic,  however.  We  foresee  future  applications  involving  online  system  identifica¬ 
tion,  detecting  sensor  spoofing,  runtime  verification,  and  enabling  a  variant  of  model- 
predictive  control  (MPC).  To  enable  these  applications,  we  are  implementing  code  gen¬ 
eration  capabilities  in  the  HYST  model  transformation  and  translation  tool  for  hybrid 
automata  [Bak  et  al.  2015],  which  will  enable  creating  implementations  of  the  real¬ 
time  reachability  algorithm  for  large  classes  of  hybrid  automata. 
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Abstract — Satisfiability  Modulo  Theories  (SMT)  solvers  have 
been  successfully  applied  to  solve  many  problems  in  formal  veri¬ 
fication  such  as  bounded  model  checking  (BMC)  for  many  classes 
of  systems  from  integrated  circuits  to  cyber-physical  systems 
(CPS).  Typically,  BMC  is  performed  by  checking  satisfiability 
of  a  possibly  long,  but  quantifier-free  formula.  However,  BMC 
problems  can  naturally  be  encoded  as  quantified  formuias  over 
the  number  of  BMC  steps.  In  this  approach,  we  then  use  decision 
procedures  supporting  quantifiers  to  check  satisfiability  of  these 
quantified  formulas.  This  approach  has  previously  been  applied 
to  perform  BMC  using  a  Quantified  Boolean  Formula  (QBF) 
encoding  for  purely  discrete  systems,  and  then  discharges  the 
QBF  checks  using  QBF  solvers.  In  this  paper,  we  present  a  new 
quantified  encoding  of  BMC  for  rectangular  hybrid  automata 
(RHA),  which  requires  using  more  general  logics  due  to  the  real 
(dense)  time  and  real-valued  state  variables  modeling  continuous 
states.  We  have  implemented  a  preliminary  experimental  proto¬ 
type  of  the  method  using  the  HyST  model  transformation  tool  to 
generate  the  quantified  BMC  (QBMC)  queries  for  the  Z3  SMT 
solver.  We  describe  experimental  results  on  several  timed  and 
hybrid  automata  benchmarks,  such  as  the  Fischer  and  Lynch- 
Shavit  mutual  exclusion  algorithms.  We  compare  our  approach 
to  quantifier-free  BMC  approaches,  such  as  those  in  the  dReach 
tooi  that  uses  the  dReal  SMT  solver,  and  the  HyComp  tool  built 
on  top  of  nuXmv  that  uses  the  MathSAT  SMT  solver.  Based  on 
our  promising  experimental  results,  QBMC  may  in  the  future  be 
an  effective  analysis  approach  for  RHA  as  further  improvements 
are  made  in  quantifier  handling  in  SMT  solvers  such  as  Z3. 

Index  Terms — bounded  model  checking,  hybrid  automata, 
timed  automata,  satisfiability  modulo  theories 

I.  Introduction 

Boolean  Satisfiability  (SAT)  is  the  canonical  NP-complete 
problem  and  is  to  determine  if  a  given  Boolean  formula 
is  satisfiable,  i.e.,  check  if  there  exists  an  assignment  of 
values  to  variables  where  the  formula  is  true.  A  Boolean 
formula  is  given  in  Conjunctive  Normal  Form  (CNF),  that 
is,  a  conjunction  of  clauses,  each  of  which  is  a  disjunction 
of  literals.  Satisfiability  modulo  theories  (SMT)  is  a  gener¬ 
alization  of  SAT,  where  literals  are  interpreted  with  respect 
to  a  background  theory  (e.g.,  linear  real  arithmetic,  nonlinear 
integer  arithmetic,  bit-vectors,  etc.). 

Recently,  SMT-based  techniques  have  been  developed  to 
formally  verify  hybrid  systems  [l]-[6].  Typically,  these  SMT- 
based  methods  are  used  in  bounded  model  checking  (BMC), 
which  is  to  check  for  a  transition  system  A  and  a  specification 
P  whether  I(s0)  A  /\^  T(si,  Sj+i)  A  (V*L0  p(si))  is  satis- 
fiable.  Here,  I(sq)  encodes  an  initial  set  of  states  over  a  set 
of  variables  So,  7'(.s,;,  .s,+  i )  represents  the  transition  relation 
from  iteration  i  to  i  +  1  over  sets  of  variables  s,  and  Sj+i,  and 
P(si)  encodes  the  specification  at  step  i. 


Hybrid  automata  are  a  modeling  formalism  used  to  verify 
dynamical  systems  including  both  continuous  states  and  dy¬ 
namics  as  well  as  discrete  states  and  transitions.  Examples 
of  systems  naturally  modeled  by  hybrid  automata  arise  in  the 
interaction  of  physical  plants  and  software  controllers  in  real¬ 
time  systems  and  cyber-physical  systems  (CPS).  In  essence, 
hybrid  automata  augment  finite  state  machines  with  a  set  of 
real-valued  variables  that  evolve  continuously  over  intervals  of 
real  time.  In  hybrid  automata,  a  transition  relation  T  =  DuT 
encodes  both  discrete  transitions  D  and  continuous  trajectories 
T  over  intervals  of  real-time.  Rectangular  hybrid  automata 
(RHA)  are  a  special  class  of  hybrid  automata  with  continuous 
dynamics  described  by  rectangular  differential  inclusions  and 
where  all  other  quantities  (guard  conditions,  invariants,  resets, 
etc.)  of  the  automata  are  linear  inequalities  over  constants  [2], 
[7].  Sets  of  states,  as  well  as  discrete  transitions  and  continuous 
trajectories  of  RHA,  can  be  symbolically  represented  by  SMT 
formulas  over  real  and  Boolean  variables. 

Depending  on  the  underlying  logics  supported,  SMT  solvers 
may  or  may  not  support  quantifiers.  While  quantifiers  make 
the  language  more  expressive,  they  increase  the  complexity 
of  computations  like  checking  satisfiability  and  may  also 
lead  to  undecidability.  Techniques  allowing  quantifiers,  such 
as  in  quantified  Boolean  formula  (QBF)  solvers,  have  been 
developed  for  BMC  of  purely  discrete  systems,  such  as  finite 
state  machines  [8],  [9].  However,  to  the  best  of  our  knowledge, 
there  has  been  no  effort  to  develop  quantified  BMC  (QBMC) 
methods  for  timed  or  hybrid  automata,  which  we  develop  in 
this  paper.  Of  course,  this  is  partially  because  the  underlying 
SMT  solver  requires  support  for  complex  combination  theories 
and  efficient  algorithms  to  check  quantified  formulas,  which 
until  recently,  were  either  not  available  or  not  scalable. 

The  logic  used  requires  some  finite  sort  for  the  discrete 
states  (such  as  a  enumerated  type  or  bitvectors)  and  reals 
for  the  continuous  states  and  trajectories.  In  this  paper,  we 
use  LRABV  (linear  real  arithmetic  with  bit-vectors)  for  en¬ 
coding  QBMC  for  timed  automata  and  RHA,  and  we  note 
that  general  hybrid  automata  would  need  NRABV  (nonlinear 
real  arithmetic  with  bit-vectors)  or  beyond,  such  as  those 
whose  solutions  involve  special  (transcendental)  functions  like 
sin,  cos,  exp,  etc.  While  none  of  these  logics  are  officially 
supported  in  the  SMT-LIB2  standard  (nor  the  2.5  draft)  as  of 
the  time  of  this  writing  [10],  several  solvers  do  have  unofficial 
support  for  this  combination  theory,  such  as  the  latest  versions 
of  Z3,  which  is  the  SMT  solver  used  in  this  paper  [11]. 
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Related  Work:  When  defining  the  semantics  of  hybrid 
automata,  first-order  or  higher  logic  is  typically  used  and 
quantifiers  typically  show  up  in  several  places.  Existential 
quantifiers  over  reals  are  used  to  specify  that  some  amount 
of  real  time  may  elapse  in  a  given  location  of  the  hybrid  au¬ 
tomaton.  Universal  quantifiers  over  reals  representing  real  time 
are  used  to  construct  invariants  that  are  enforced  at  all  times, 
while  in  a  given  location  of  the  hybrid  automaton;  otherwise 
real  time  is  not  allowed  to  advance,  and  a  discrete  transition 
must  be  taken,  if  any  are  enabled  based  on  the  current  state 
and  guards  of  the  transitions.  Alternative  approaches  to  the  one 
described  in  this  paper  have  previously  been  developed,  where 
the  universal  quantifiers  used  to  define  invariants’  semantics 
are  explicitly  removed  from  the  SMT  expressions  to  create 
quantifier-free  formulas.  This  allows  the  use  of  existing  SMT- 
based  procedures  and  avoids  quantifier-elimination  and  other 
quantifier-handling  procedures  [2],  [3],  [12].  We  note  that  this 
approach  does  not  use  quantifiers  on  the  number  of  steps 
k  >  0  in  the  BMC  computation,  which  we  do  in  this  paper. 
Specifically,  we  suggest  that  effectively  encoding  the  BMC 
problem  in  a  quantified  form  over  the  number  of  steps  k  may 
provide  a  more  scalable  approach  in  the  future  as  quantifier 
handling  procedures  are  improved  in  the  underlying  solvers. 
We  accomplish  this  by  extending  existing  results  for  BMC  of 
discrete  systems  with  QBF  solvers  [9]  to  timed  and  hybrid 
automata,  specifically  RHA. 

Typical  approaches  to  analyze  timed  and  hybrid  automata 
use  symbolic  representations  of  states  such  as  difference  bound 
matrices  (DBMs)  to  represent  clock  regions  in  Uppaal  [13]  or 
polyhedra  in  HyTech  [14],  Several  other  formal  verification 
tools  for  hybrid  automata  focus  on  performing  reachability 
computations,  and  overapproximate  the  set  of  reachable  states 
using  various  data  structures  to  symbolically  represent  geo¬ 
metric  sets  of  states,  such  as  Taylor  models  in  Flow*  [15]  and 
support  functions  in  SpaceEx  [16].  Reachability  analysis  tools 
like  Flow*  and  SpaceEx  focus  on  computing  reachable  states, 
although  there  is  a  direct  equivalence  between  time-bounded 
reachability  computations  and  BMC. 

Several  SMT-based  approaches  can  verify  properties  of 
timed  and  hybrid  automata.  dReal  is  an  SMT-solver  for  first- 
order  logic  formulas  over  the  reals,  and  uses  a  ^-complete 
decision  procedure  [17].  dReach  is  a  BMC  tool  that  queries 
dReal  to  check  satisfiability  of  SMT  formulas  encoding  the 
transitions  and  trajectories  for  hybrid  automata  [4],  HyComp 
is  a  verification  tool  for  networks  (parallel  compositions)  of 
hybrid  automata  with  polynomial  and  other  dynamics  [6]  and 
is  built  on  top  of  nuXmv  [18].  For  fc-induction  and  IC3, 
HyComp  may  perform  unbounded  model  checking,  but  in  the 
BMC  mode,  it  also  allows  a  limit  on  the  number  of  steps, 
and  also  encodes  the  semantics  of  the  network  of  hybrid 
automata’s  transition  relation  and  trajectories.  A  very  closely 
related  approach  to  this  paper  also  encodes  BMC  problems 
for  timed  automata  using  quantified  formulas,  but  this  quan¬ 
tification  is  to  encode  unknown  or  incomplete  components, 
and  is  not  a  quantification  over  the  BMC  length  [19].  Passel 
is  a  parameterized  verification  tool  for  networks  of  RHA 


that  may  prove  properties  regardless  of  the  number  N  of 
automata  in  the  network  [2],  Passel  implements  an  extension 
to  hybrid  automata  of  the  invisible  invariants  approach  for  pa¬ 
rameterized  verification,  and  consists  of  an  invariant  synthesis 
procedure  [20]  that  relies  on  reachability  computations  [5]. 
Passel  encodes  the  semantics  of  networks  of  hybrid  automata 
as  SMT  formulas  and  checks  satisfiability  and  validity  using 
the  Z3  SMT  solver.  Additionally,  when  performing  reachabil¬ 
ity  computations,  Passel  makes  use  of  quantifier  elimination 
procedures  over  the  reals  and  bit-vectors  [5]. 

Contributions:  In  this  paper,  we  present  a  new  SMT- 
based  verification  technique  that  encodes  the  BMC  problem 
for  RHA  in  a  quantified  form,  which  we  call  quantified 
BMC  (QBMC).  We  take  hybrid  automata  in  the  SpaceEx 
format  [16],  which  are  then  translated  to  the  QBMC  encoding 
proposed  in  this  paper  using  the  HyST  model  transformation 
tool  [21].  We  then  perform  QBMC  by  querying  the  Z3  SMT 
solver  via  its  Python  API  and  use  its  quantifier-handling  proce¬ 
dures  [11],  We  present  preliminary  experimental  results  where 
the  QBMC  approach  and  Z3  perform  competitively,  when 
compared  to  (a)  the  dReach  tool  that  performs  BMC  using  an 
SMT  check  by  querying  the  dReal  ^-decidable  SMT  solver  [4], 
[17],  and  (b)  the  HyComp  tool  built  on  top  of  nuXmv  that  uses 
the  MathS  AT  SMT  solver  [22].  The  examples  include  standard 
ones  such  as  Fischer  and  Fynch-Shavit  mutual  exclusion,  as 
well  as  an  illustrative  example  to  describe  the  encoding.  The 
main  contribution  of  this  paper  is  the  first  encoding  of  BMC  as 
a  quantified  problem  for  RHA.  Our  results  subsume  the  case 
for  timed  automata,  as  RHA  are  more  expressive  than  timed 
automata,  and  we  note  this  is  also  the  first  QBMC  approach 
for  timed  automata. 

II.  Hybrid  Automata  Syntax  and  Semantics 

A  hybrid  automaton  is  essentially  a  finite  state  machine 
extended  with  a  set  of  real-valued  variables  that  evolve  con¬ 
tinuously  over  intervals  of  real-time. 

Syntax:  The  syntactic  structure  of  a  hybrid  automaton  is 
formally  defined  as  follows. 

Definition  1:  A  hybrid  automaton  77,  is  a  tuple,  77  = 
( Loc ,  Far,  Inv ,  Flow,  Trans,  Init),  with  the  components  as 
follows,  (a)  Loc  is  a  finite  set  of  discrete  locations,  (b)  Var 
is  a  finite  set  of  n  continuous,  real-valued  variables,  and 
Q  =  Loc  x  K™  is  the  state-space,  (c)  Inv  is  a  finite  set 
of  invariants,  one  for  each  discrete  location,  and  for  each 
location  I  £  Loc,  Inv  {I )  C  Rn.  (d)  Flow  is  a  finite  set 
of  ordinary  differential  inclusions,  one  for  each  continuous 
variable  x  £  Var,  and  Flow{l,x )  C  R™  describes  the 
continuous  dynamics  in  each  location  £  £  Loc.  (e)  Trans  is 
a  finite  set  of  transitions  between  locations.  Each  transition  is 
a  tuple  t  =  (£,£',  g,u),  where  £  is  a  source  location  and  £’ 
is  a  target  location  that  may  be  taken  when  a  guard  condition 
g  is  satisfied,  and  the  post-state  is  updated  by  an  update  map 
u.  (f)  Init  is  an  initial  condition,  which  consists  of  a  set  of 
locations  in  Loc  and  a  formula  over  Var,  so  that  Init  C  Q. 

For  RHA,  all  the  expressions  appearing  in  invariants, 
guards,  and  updates  must  be  boolean  combinations  of  constant 
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inequalities,  and  the  flows  are  rectangular  differential  inclu¬ 
sions  ( x  £  [a,  b]  for  a  <  b)  [7],  We  use  the  dot  (.)  notation 
to  refer  to  different  components  of  tuples  e.g.,  77, .  Inv  refers 
to  the  invariants  of  automaton  77  and  r.g  refers  to  the  guard 
of  a  transition  r.  If  clear  from  context,  we  drop  77  and  r  and 
refer  to  the  individual  components  of  the  tuple. 

Semantics:  The  semantics  of  a  hybrid  automaton  77  are 
defined  in  terms  of  executions,  which  are  sequences  of  states. 
A  state  q  of  77  is  a  tuple  q  =  (7,  v),  where  7  £  Loc  is  a 
location,  and  v  £  Si'"  is  a  valuation  of  all  variables  in  Var. 
Formally,  for  a  set  of  variables  Var,  a  valuation  is  a  function 
mapping  each  x  £  Var  to  a  point  in  its  type — here.  Si.  The 
state-space  Q  is  the  set  of  all  states  of  77.  Updates  of  states 
are  described  by  a  transition  relation  T  C  Q  x  Q.  For  a 
transition  ( q,q ')  £  T  where  q  =  (£,v)  and  q'  =  (£',v'), 
we  denote  q  — >  q’  £  T  as  the  transition  between  the 
current  state  q  and  the  next  state  q' .  The  transition  relation 
T  is  partitioned  into  disjoint  sets  of  discrete  transitions  and 
continuous  trajectories  that  respectively  describe  the  discrete 
and  continuous  behaviors  of  the  automaton.  Thus,  T  =  DuT, 
where:  (a)  D  C  Q  x  Q  is  the  set  of  discrete  transitions  that 
describe  instantaneous  updates  of  state,  (b)  T  C  Q  x  Q  is 
the  set  of  continuous  trajectories  that  describe  updates  of  state 
over  real  time  intervals. 

Discrete  transitions.  A  discrete  transition  q  -A  q1  £  I) 
models  an  instantaneous  update  from  the  current  state  q  to 
the  next  state  q'.  There  is  a  discrete  transition  q  -A  q'  £  I)  if 
and  only  if  (iff):  3 r  £  Trans  :  q.v  \=  r.gAq' .v'  \=  t.u,  where 
r.g,  and  t.u  are  the  guard  condition  and  the  update  map  of 
the  discrete  transition  r,  respectively. 

Continuous  trajectories.  A  continuous  trajectory  q  -A  q'  £ 
T  models  the  update  of  state  q  to  q'  over  an  interval  of  real 
time.  The  set-valued  function  A  returns  a  set  of  states  and  is 
defined  as:  A(q.£,q.v,x,t)  £  q.v. x  +  Js=tof(q.£,x)d6,  where 
/  £  Flow  is  a  flow  rate.  A  formula  over  Var  U  Var  that 
describes  the  evolution  of  a  real  variables  x  £  Var  over  a  real 
time  interval  J  =  [7o,7],  and  q-v.x  is  the  value  of  continuous 
variable  x  of  the  state  q  at  t  =  to-  Then,  there  is  a  trajectory 
q  — t  (f  £  T  iff:  3 t-a  £  M>q  Vtp  £  M>q  37  £  Loc  : 
tp  <  ta  A  A (q.£,q.v,  Var,tp)  \=  Inv{ 7)  A  q'.v'.Var  £ 
A {q.£,q.v,  Var,ta).  For  each  real  variable  x,  q.v.x  must 
evolve  to  the  valuation  q'.v'. x  at  precisely  time  ta  and 
corresponding  to  the  flow  rate  of  x  in  location  7.  Additionally, 
all  states  along  the  trajectory  must  satisfy  the  invariant  Inv( 7) 
i.e.,  at  every  point  in  the  interval  of  real  time  tp  <ta. 

Executions.  An  execution  of  77  is  a  sequence  7r  =  go  —■ ► 
7i  — >  72  — i ►  •  such  that:  (a)  q o  £  Init  is  an  initial  state, 
and  (b)  either  qt  — >  qi+i  £  D  is  a  discrete  transition  or  qi  — t 
7i+i  £  T  is  a  continuous  trajectory  for  each  consecutive  pair 
of  states  in  the  sequence  n.  A  state  qk  =  (£k,  Vk)  is  reachable 
from  initial  state  go  =  ( 7o ,  vq)  £  Init  iff  there  exists  a  finite 

A 

execution  n  =  qo  — I ►  q\  >  ...  — I *  qk- 

Safety  specifications .  In  this  paper,  we  develop  the  QBMC 
procedure  to  check  whether  safety  properties  of  hybrid  au¬ 
tomata  are  satisfied  up-to  iteration  k.  A  safety  specification 


f  is  a  formula  over  Loc  and  Var  that  describes  a  set  of 
states  [0]  C  Q,  where  [■]  is  the  set  of  states  satisfying  <j>.  For 
an  automaton  77  and  a  safety  specification  f,  the  automaton 
satisfies  the  specification,  denoted  ji  \=  f,  iff  for  every 
execution  tt,  for  every  state  qo,qi,  ■  ■  ■  ,qk  in  the  execution  ir, 
we  have  n .qk  £  [</>].  If  77  |=  <f>  for  every  i  £  {0, . . . ,  k}, 
then  the  system  is  safe  up-to  iteration  A:.  If  77  |=  for  any 
k,  then  the  system  is  safe.  For  a  safety  specification  <f>,  a 
counterexample  is  an  execution  7r  where  some  state  q  £  n 
violates  <f>,  i.e.,  q  </>,  or  equivalently,  q  (j  [</>]. 

III.  Quantified  BMC  for  Hybrid  Automata 

Bounded  model  checking  (BMC)  has  been  used  widely  in 
verification  and  falsification  of  safety  and  liveness  properties 
of  various  classes  of  systems,  from  finite  state  machines  to 
hybrid  automata.  The  key  idea  is  to  search  for  a  counterex¬ 
ample  execution  whose  length  is  bounded  by  a  number  of 
steps  k.  In  other  words,  BMC  will  explore  all  executions 
from  any  initial  state  of  the  system  T  to  detect  whether  there 
is  a  way  to  reach  a  bad  state  that  violates  a  given  property 
(or  to  find  a  loop  in  the  case  of  liveness).  Then  this  path  is 
considered  as  a  counterexample  to  the  property  that  may  help 
the  user  to  debug  the  system.  For  finite  state  systems,  BMC 
can  be  encoded  as  a  propositional  formula  to  be  checked  as 
satisfiable  or  unsatisfiable  using  a  Boolean  SAT  solver.  For 
hybrid  automata,  BMC  can  be  encoded  as  a  formula  over  reals 
and  finite  sorts  (such  as  Booleans,  bitvectors,  or  enumerated 
types).  In  this  paper,  we  focus  only  on  hybrid  automata  with 
rectangular  differential  inclusion  dynamics  (x  £  [a,  b]  for  real 
constants  a  <  b),  and  for  this  class  of  automata,  the  formulas 
are  within  linear  real  arithmetic  (LRA).  We  first  illustrate 
BMC  for  hybrid  automata  using  the  traditional  quantifier-free 
encoding,  and  then  describe  the  quantified  BMC  (QBMC), 
which  is  the  main  contribution  of  this  paper. 

Quantifier-Free  BMC  for  Hybrid  Automata:  Let  P  be  a 
set  of  given  specifications  of  the  hybrid  automata,  the  BMC 
problem  will  determine  whether  a  specification  P{qk)  £  P  is 
safe  after  k  steps,  and  it  is: 

k  —  1  k 

$(fc)  =  /(U0)A  f\Ti{V,V')  P{Vff),  (1) 

2—0  2—0 

where  V,  corresponds  to  the  set  of  variables  Var  of  the 
automaton  77  appropriately  renamed.  For  example,  Vt  contains 
of  every  variable  v  £  Var  syntactically  renamed  to  v.t, 
etc.,  and  V'  consists  of  primed  variables,  e.g.,  v'  for  each 
v  £  Var.  In  Equation  1,  /(Vo)  encodes  the  initial  set  of 
states,  Ti{V,V')  encodes  the  transition  between  consecutive 
pairs  of  sets  of  states,  and  P(Vi )  is  a  safety  specification 
at  iteration  i.  We  note  that  the  sets  of  variables  Vi  for  each 
iteration  i  are  implicitly  existentially  quantified  and  e.g.,  we 
could  equivalently  prefix  3  Vo,  V±, ...  ,Vk-  We  drop  the  sets  of 
variables  for  a  shorter  notation,  e.g..  Equation  1  is  equivalent 

to/oAAto^AjVLo^)- 

Example  1:  Consider  the  hybrid  automaton  77  shown  in  Fig¬ 
ure  1.  Assume  that  the  automaton  starts  at  location  7 ocj,  and 
the  initial  value  of  x  is  0.  The  set  of  bad  states  are  defined 
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by:  P  =  Vi=o  =  V-oc2  =>  a:  >  2.5).  Two 

intervals  [ai,6i]  and  [02,62]  describe  the  rectangular  differ¬ 
ential  inclusions  for  locations  toci,  and  1 0C2 ,  respectively. 
This  automaton  would  be  a  timed  automaton  if  all  of  the 
constants  values  are  equal,  i.e.,  a±  =  61  =  02  =  62.  This 
automaton  would  be  a  multi-rate  timed  automaton  if  ai  =  b\ 
and  02  =  62  but  possibly  01  /  02.  Otherwise,  this  automaton 
is  a  rectangular  hybrid  automaton.  Suppose  that  ai  =  1, 
bi  =  2,  02  =  3,  and  b2  =  4.  We  introduce  k  +  1  copies 
xo,Xi,  ...,Xfc  and  £o,£i,  ...,£h,  where  the  variable  x$  gives  the 
value  of  the  variable  x,  and  £i  indicates  the  location  at  the 
state  qt,  representing  the  Ith  step  of  the  BMC  computation  for 
the  automaton  shown  in  Figure  1.  The  BMC  computation  of 
H  for  each  k  up  to  2  can  be  encoded  as: 

•  k  =  0:  Iq  :=  (To  =  £oci  A  xq  =  0); 

•  k  =  1  (Do):  (£0  =  £oci  A  £\  =  £002  A  xo  <  5  A  xo  > 
2.5  A  Xi  =  Xo), 

•  k  =  1  (To)'-  {£ 0  =  £oci  =>  (l\  =  £0  A  Xo  +  or S  < 
Xi  A  xi  <  Xo  +  biS  A  Xi  <  5)), 

•  k  =  2  (DQ:  (fT  =  f'ocj  A  £2  =  £002  Axj  <  5  Axi  > 
2.5  A  X2  =  Xi), 

•  k  =  2  (71):  (f'l  =  £oci  =>  (£2  =  4  A  ii  +  016  < 
X2  A  X2  <  Xi  +  616  A  x2  <  5)), 

where  6  is  a  fresh,  real  constant.1  We  split  the  discrete 
transitions  and  trajectories  for  clarity,  but  the  entire  formula  to 
be  checked  for  iteration  k  =  1  would  just  be  the  disjunction 
of  these  conjuncted  with  the  formula  representing  k  =  0  and 
the  bad  set  of  states,  i.e.,  Jo  A  (Do  V  To)  A  P.  For  k  =  2,  this 
full  formula  would  be  Iq  A  (Do  V  7o)  A  (Di  V  71)  A  P. 

For  k  =  1,  we  dropped  the  obviously  infeasible  transition 
from  £002  to  toci  from  Do,  which  would  be  found  as  being 
unsatisfiable  since  to  £002-  However,  the  transition  from 
loci  to  £002  also  cannot  occur  since  xo  =  0,  but  Xo  ^  2.5, 
so  that  part  is  unsatisfiable  and  no  discrete  transitions  may 
be  taken  from  the  set  of  initial  states.  We  also  dropped  the 
continuous  dynamics  for  £ocg  from  To  since  this  would  also 
be  infeasible  since  £0  7^  £002-  However,  real  time  may  elapse, 
and  as  encoded,  would  correspond  to  any  choice  of  time  5  such 
that  xi  £  [a i(S,  616]  and  xi  <  5.  Since  a  1  =  1  and  bi  =  2, 
at  most  between  2.5  and  5  seconds  of  real  time  could  elapse, 
and  either  case  would  yield  xi  £  [0,  5], 

For  k  =  2,  we  also  dropped  the  infeasible  transition  and 
trajectory  for  clarity.  In  this  case,  the  transition  from  toci 
to  £002  is  enabled  since  xi  £  [0,5],  so  the  update  to  £002 
may  occur.  However,  now  the  continuous  trajectory  would 
be  infeasible  since  Xi  could  already  be  5  and  the  invariant 
requires  x2  <  5,  so  no  real-time  6  >  0  may  elapse,  as 
otherwise  xi+aiS  >  5  is  unsatisfiable  for  xi  =  5.  So,  the  only 
state  update  would  be  to  I0C2  owing  to  the  discrete  transition. 

1  In  general,  a  universally  quantified  assertion  that  the  invariant  is  satisfied 
for  every  real  time  along  the  trajectory  from  time  to  to  time  to  +  <5,  although 
this  is  unnecessary  for  rectangular  differential  inclusions  with  linear  guards 
and  invariants  for  convexity  reasons  [2],  [6],  which  makes  this  assertion  fall 
into  the  combination  theory  of  linear  real  arithmetic  with  bitvectors  (or  some 
finite  sort  to  encode  the  locations). 


x  >  2.5 


Fig.  1 .  The  hybrid  automaton  "H  for  Example  1 . 

Quantified  BMC  (QBMC)  for  Hybrid  Automata:  Next,  we 
construct  a  quantified  formula  f l(k)  for  BMC  of  77,  of  length 
k.  We  introduce  a  vector  t  =  (t\,t2, ...,  f  [iog2  x\ )  to  index  each 
iteration  of  the  BMC  of  77.  The  current  state  q  and  next  state 
q '  under  the  transition  relation  T(  V,  V)  are  connected  to  the 
current  state  and  the  next  state  for  each  particular  iteration  U, 
for  i  £  [1,  [log2  &]].  The  quantified  BMC  formula  is: 

fi(fc)  =  3Vo,  Vj, ...,  Vfc,  <5Vt3V,  V'  \  7(H0)  A  T(V,  V')  A 

k—  1  k 

A  Kv  =  K) A  (Yf  =  vi+1)]  a  (\/  p(Vi)), 

i= 0  2—0 

where  we  note  that  the  existential  5  encodes  the  real  time 
elapse  and  would  appear  in  the  trajectories  T  of  the  disjunct 
T  =  DMT. 

For  k  =  3,  the  QBMC  of  the  automaton  of  Example  1  is: 
fi(3)  =  3 Vo,  Vi,  V2,  V3,  <5Vfr,  t23V,  V'  \  I(V0)  A  T(V,  V') 

A  {fj  -»■  [(V  =  Vj,)  A  (V1  =  Hi)]} 

A  {ii  A  t2  -A  [(H  =  Hi)  A  (H'  =  H2)]} 

A  {h  A  t2  ->  [(H  =  H2)  A  (H'  =  H3)]| 

A  (P(H0)  VP(Hi)  VP(H2)  VP(H3)),  (2) 

where  H  =  V'  is  a  shorthand  indicating  every  variable  v  £  V 
equals  its  corresponding  counterpart  v'  £  H.  In  Equation  2, 
if  the  value  of  t\  is  0,  then  there  is  a  continuous  trajectory 
that  evolves  from  the  initial  state  q0,  where  qo-£o  =  toci 
and  Xo  =  0,  to  the  next  state  qi,  where  q \.£\  =  toci  and 
xi  <  5.  When  t\  =  1  and  t2  =  0,  the  system  takes  the 
discrete  transition  from  the  current  state  <j\  to  the  next  state 
q2,  where  q2.£2  =  £ 0C2  and  the  value  of  x3  is  not  higher  than 
10.  At  k  =  3,  both  ii,  and  t2  are  true,  then  q2  becomes  the 
current  state,  and  q3  is  the  next  state,  where  q373  =  £oci , 
and  x3  <  5.  The  discrete  transition  taken  from  q2  to  f/3  when 
x  >  10  will  reset  the  value  of  x  to  0. 

If  it  terminates,  an  SMT  solver  supporting  the  combined 
theory  of  bitvectors  and  reals  with  quantifiers  will  return  SAT 
for  the  QBMC  formula  iff  there  exists  an  execution  from  an 
initial  state  to  a  bad  state,  i.e.,  if  a  bad  state  is  reachable. 
Otherwise,  if  it  terminates,  it  will  return  UNSAT  if  a  bad  state 
is  not  reachable  in  k  steps.  We  note  that  the  combination  theory 
of  linear  real  arithmetic  with  bitvectors  is  decidable,  and  Z3 
is  in  essence  a  decision  procedure  for  this  theory. 

IV.  Experimental  Results 

We  implement  the  method  described  in  this  paper  as  a 
module  within  HyST  [21].  HyST  takes  as  input  a  hybrid 
automaton  model  in  an  extended  form  of  the  SpaceEx  XML 
format  [16]  (supporting  e.g.,  nonlinear  functions  instead  of 
only  affine  ones),  and  creates  the  transition  relation  as  SMT 
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TABLE  I 

Example  1  performance  comparison. 


Tools 

L 

k  <  32 

k  <  64 

k  <  128 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

QBMC 

2 

1.11 

27.2 

3.68 

39.4 

19.9 

91.2 

dReach 

2 

86.7 

102.4 

1176.4 

284.7 

20034 

829.2 

HyComp 

2 

0.4 

97.3 

0.6 

101.8 

1.44 

109.3 

formulas  using  the  Z3  Python  API.  We  evaluate  the  QBMC 
method  described  in  this  paper  on  several  examples.2  We 
compare  the  results  from  the  QBMC  method  of  this  paper 
with  that  of  dReach,  which  is  a  state-of-the-art  BMC  tool  for 
nonlinear  hybrid  automata  [23],  and  with  that  of  HyComp 
that  uses  the  MathS AT  SMT  solver  [6].  All  of  the  models 
for  dReach  and  HyComp  are  also  generated  using  HyST.  The 
experiments  are  performed  on  Intel  15  2.4GHz  processor  with 
3GB  RAM,  executing  the  method  described  in  this  paper  and 
dReach  in  a  VirtualBox  virtual  machine  running  Ubuntu  64- 
bit.  Z3  version  4.3.2  was  used  in  the  evaluation.  We  collect  the 
running  times  (Time)  in  seconds  and  the  peak  memory  usages 
(Mem)  in  megabytes  for  different  examples. 

We  first  evaluate  our  QBMC  encoding  on  the  illustrative 
hybrid  automata  presented  in  Example  1,  and  compare  the 
results  to  those  of  dReach  and  HyComp.  The  performances 
of  those  three  different  methods  are  shown  in  Table  I,  where 
QBMC  denotes  the  QBMC  presented  in  this  paper,  k  is  a 
number  of  steps  in  the  BMC  computation,  and  L  is  the  number 
of  discrete  locations.  The  constants  values  are  given  as:  a,\  = 
0,  bi  =  1,02  =  0,  and  b2  =  2.  The  results  shown  in  Table  I 
preliminarily  indicate  that  our  QBMC  approach  is  capable  of 
solving  BMC  significant  faster  than  dReach,  but  slower  than 
HyComp.  However,  our  approach  requires  less  memory  usage 
compared  to  dReach  and  HyComp. 

Next,  we  evaluate  QBMC  with  several  scenarios  using 
the  Fischer  mutual  exclusion  protocol  [2],  Fischer  mutual 
exclusion  is  a  timed  distributed  algorithm  that  ensures  a  mutual 
exclusion  safety  property,  namely  that  at  most  one  process 
in  a  network  of  N  processes  may  enter  a  critical  section 
simultaneously.  The  set  of  bad  states  is  defined  by: 

(j)  =  -Ni,j  S  {1, . . . ,  N}  I  (i  ^  j  A  qi  =  cs)  -A  qj  ^  cs, 
where  qi  and  qj  are  variables  modeling  the  discrete  location 
of  the  automata,  cs  is  the  critical  section  location,  and  — >  is 
logical  implication.  We  compare  the  performance  of  QBMC 
in  solving  the  BMC  of  Fischer  protocol  with  HyComp  and 
dReach.  Figures  2  and  3  show,  respectively,  the  runtime 
and  memory  usage  comparison  among  HyComp,  dReach  and 
QBMC  for  different  numbers  of  processes  of  Fischer  protocol; 
where  QBMC-safe,  QBMC-unsafe,  HyComp-safe,  HyComp- 
unsafe,  dReach-safe,  and  dReach-unsafe  denote  the  BMC 
of  the  safe  and  unsafe  version  of  Fischer  protocol  using 
QBMC,  HyComp,  and  dReach,  respectively.  Overall,  HyComp 
is  generally  faster  than  QBMC.  However,  it  requires  a  higher 
memory  consumption  than  QBMC.  For  instance,  with  k  <  16, 

-The  preliminary  implementation  described  in  this  paper,  along  with  all  the 
examples,  is  available  online  at:  http://www.verivital.com/hyst/cfv2015.zip. 


TABLE  II 

Lynch-Shavit  mutual  exclusion  protocol  performance. 


Tools 

L 

k  <  4 

k  <  8 

k  <  16 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

QBMC 

92 

3.7 

52.2 

5.1 

52.3 

25.9 

52.7 

93 

15.5 

65.6 

31.3 

87.5 

1091.5 

144.5 

94 

256.1 

702.8 

1062.1 

708.9 

43578 

1196.2 

HyComp 

92 

0.8 

121.9 

1.33 

132.8 

9.5 

170.5 

93 

2.7 

307.9 

12.81 

380.8 

192.8 

771.4 

94 

63.9 

2655.4 

N/A 

M/O 

N/A 

M/O 

the  BMC  of  the  unsafe  version  of  Fischer  protocol  with  5 
processes  cannot  terminate  in  HyComp  due  to  out  of  memory 
(requiring  more  than  3GB).  However,  QBMC  can  solve  it 
using  less  than  500  MB.  Thus,  we  can  point  out  that  QBMC 
is  superior  than  HyComp  with  respect  to  the  memory  usage. 
Moreover,  Figures  2  and  3  also  indicate  that  QBMC  is  able  to 
solve  BMC  of  hybrid  automata  faster  and  uses  less  memory 
than  dReach.  Due  to  state-space  (and  formula)  explosion, 
the  reduction  of  memory  consumption  is  one  of  the  major 
challenges  to  address.  Since  QBMC  requires  a  smaller  amount 
of  memory  usage  than  other  quantifier-free  BMC  approaches, 
it  may  be  effective  in  solving  BMC  of  large  scale  problems. 

We  also  evaluate  QBMC  with  the  Fynch-Shavit  mutual 
exclusion  protocol.  The  Fynch-Shavit  protocol  is  a  modi¬ 
fied  version  of  Fischer  protocol  where  the  mutual  exclu¬ 
sion  property  is  time-independent.  Each  process  of  Fynch- 
Shavit  protocol  has  9  states  (locations),  then  the  Fynch-Shavit 
protocol  with  4  processes  includes  6561  discrete  locations. 
The  performance  analyzing  the  Fynch-Shavit  protocol  using 
QBMC  and  Hycomp  are  shown  in  Table  II,  respectively.  M/O 
presents  that  the  peak  memory  usage  is  higher  than  3GB, 
and  N/A  denotes  that  the  information  of  running  times  is 
not  detected  due  to  M/O.  The  set  of  bad  states  of  Fynch- 
Shavit  protocol  is  defined  similar  to  that  of  Fischer,  where 
two  processes  may  be  in  the  critical  section.  Again,  we  can 
see  the  trade  off  between  using  QBMC  or  using  HyComp. 
HyComp  is  faster  than  QBMC,  but  requires  a  higher  memory 
usage.  Therefore,  the  BMC  of  Fynch-Shavit  protocol  with  4 
processes  can  be  solved  by  QBMC  up  to  k  =  16,  but  cannot 
be  solved  in  HyComp  up  to  k  =  8  due  to  M/O. 

V.  Conclusion  and  Future  Work 

In  this  paper,  we  present  a  new  SMT-based  technique 
that  encodes,  in  a  quantified  form,  the  BMC  problem  for 
rectangular  hybrid  automata  (RHA),  which  also  subsumes  this 
encoding  for  timed  automata.  The  preliminary  results  for  the 
Fischer  mutual  exclusion  protocol  and  Fynch-Shavit  protocol 
indicate  the  capability  of  our  method  to  solve  the  BMC 
problem  for  hybrid  systems  including  more  than  a  thousand 
locations.  We  compare  these  experimental  results  to  those  of 
quantifier-free  BMC  approaches,  such  as  in  the  dReach  tool 
that  uses  the  dReal  SMT  solver,  and  the  HyComp  tool  built 
on  top  of  nuXmv  that  uses  the  MathSAT  SMT  solver.  As 
solvers  for  fragments  of  many-sorted  first-order  logic  such  as 
FRA,  NRA,  etc.  continue  to  improve,  QBMC  encodings  such 
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QBMC-safe  --a--  QBMC-unsafe  l  -  HyComp-safe  0  HyComp-unsafe  ---A-*  dReach-safe  dReach-unsafe 


Number  of  Processes  Number  of  Processes  Number  of  Processes  Number  of  Processes 


Fig.  2.  Runtime  comparison  of  HyComp,  dReach  and  QBMC  in  solving  the  BMC  of  Fischer  protocol. 


k<  4  k  <  8  k  <  16  k  <  32 


Number  of  Processes  Number  of  Processes  Number  of  Processes  Number  of  Processes 


Fig.  3.  Memory  usage  comparison  of  HyComp,  dReach  and  QBMC  in  solving  the  BMC  of  Fischer  protocol. 


as  the  one  described  in  this  paper  will  become  more  effective, 
similar  to  how  QBMC  for  discrete  systems  has  been  shown 
to  be  effective  with  QBF  encodings  [9],  In  future  work,  we 
will  conduct  additional  experiments  and  compare  the  results 
to  other  tools  and  techniques,  such  as  UPPAAL,  and  also 
investigate  more  general  classes  of  hybrid  automata,  such  as 
those  with  linear  or  polynomial  differential  equations. 
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Appendix 

A.  Appendix:  Additional  Experimental  Results 

In  this  appendix,  we  describe  additional  experimental  results 
of  the  BMC  of  the  Fischer  mutual  exclusion  protocol  using 
QBMC,  HyComp  and  dReach.  Figures  4  and  5  show,  re¬ 
spectively,  the  runtime  and  memory  usage  comparison  among 
HyComp,  dReach  and  QBMC  for  BMC  of  Fischer  protocol. 
Vertical  axises  are  runtime  in  seconds  and  memory  usage  in 
megabytes,  respectively,  and  horizontal  axises  are  number  of 
steps,  k.  The  details  of  running  times  and  memory  usages  of 
BMC  for  the  Fischer  protocol  using  these  tools  are  also  shown 
in  Table  III,  where  FS,  FU  denote  the  safe  and  unsafe  versions 
of  Fischer  protocol,  respectively,  and  the  number  following  the 
hyphen  (-)  describes  a  number  of  processes  for  each  version. 
In  FS,  a  state  where  the  set  of  bad  states  <fi  is  satisfied  is  not 


reachable,  while  in  FU,  a  state  where  <f>  is  satisfied  is  reachable. 
For  instance,  FS-2,  FU-2  are  the  safe  and  unsafe  versions  of 
the  Fischer  protocol  with  2  processes,  respectively. 

Table  III  shows  that  the  BMC  of  Fischer  protocol  with  64 
discrete  locations  can  be  checked  completely  up  to  k  =  32. 
Note  that  T/O  means  the  computation  time  out  (>  24  hours), 
M/O  presents  that  the  peak  memory  usage  is  higher  than  3GB, 
and  N/A  denotes  that  the  information  of  times  or  memory 
usages  are  not  detected  due  to  M/O  or  T/O,  respectively.  The 
results  of  the  BMC  for  unsafe  versions  of  Fischer  protocol 
indicate  that  QBMC  is  effective  for  bug  detection.  However, 
as  k  increases,  the  higher  running  time  and  the  greater  memory 
usage  are  required  for  the  quantified  encoding  of  BMC  due 
to  the  increasing  number  of  all  possible  paths  from  an  initial 
state  in  the  set  of  initial  states  to  a  bad  state  that  does  not 
satisfy  the  set  of  safety  specifications. 
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Memory  Usage  (MB)  Runtime  (s) 


QBMC-safe  --B--  QBMC-unsafe 


HyComp-safe 


HyComp-unsafe  ■■■A-**  dReach-safe  dReach-unsafe 


Fig.  4.  Runtime  comparison  of  HyComp,  dReach  and  QBMC  in  solving  the  BMC  of  Fischer  protocol. 


Fig.  5.  Memory  usage  comparison  of  HyComp,  dReach  and  QBMC  in  solving  the  BMC  of  Fischer  protocol. 


TABLE  III 

The  performance  of  the  BMC  of  Fischer  mutual  exclusion  protocol  using  QBMC,  HyComp,  and  dReach. 


Tools 

Example 

L 

k  <  4 

k  <  8 

k  <  16 

k  <  32 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

Time 

(sec) 

Mem 

(MB) 

QBMC 

FS-2 

42 

1.11 

22.3 

1.6 

25.2 

6.4 

30 

60 

45.2 

FU-2 

42 

0.7 

21.73 

1.1 

24.7 

1.52 

28.2 

6.1 

40.2 

FS-3 

43 

4.02 

48.7 

8.3 

48.7 

117.8 

52.4 

19452 

115.6 

FU-3 

43 

3.97 

48.7 

6.9 

48.7 

22.7 

49.7 

94.3 

74.6 

FS-4 

44 

9.97 

56.9 

76.1 

74.1 

T/O 

N/A 

T/O 

N/A 

FU-4 

44 

8.44 

57 

40.1 

73.2 

119.1 

156.2 

4197.1 

254.1 

FS-5 

45 

77.51 

254.3 

344.4 

254.4 

T/O 

N/A 

T/O 

N/A 

FU-5 

45 

63.93 

249.9 

288.8 

249.9 

21456 

473.8 

T/O 

N/A 

HyComp 

FS-2 

42 

0.2 

22.3 

0.5 

101.4 

2.8 

107.3 

14.1 

123.4 

FU-2 

42 

0.2 

21.7 

0.4 

100.9 

0.5 

101.4 

0.53 

101.5 

FS-3 

43 

0.51 

120.2 

2.2 

131.8 

55.8 

214.4 

539.7 

713.4 

FU-3 

43 

0.51 

121.5 

2.1 

131.8 

6.7 

149.6 

6.5 

167.1 

FS-4 

44 

2.78 

255 

9.9 

319.1 

788 

1010.4 

T/O 

M/O 

FU-4 

44 

2.53 

255.2 

13.3 

318.2 

569.4 

895.4 

568.4 

897.1 

FS-5 

45 

17.13 

1067 

172.4 

1405.9 

N/A 

M/O 

N/A 

M/O 

FU-5 

45 

16.6 

1066.7 

109.1 

1345.4 

N/A 

M/O 

N/A 

M/O 

dReach 

FS-2 

42 

1.2 

2.5 

64.1 

120.8 

T/O 

M/O 

T/O 

M/O 

FU-2 

42 

1.2 

2.5 

48.4 

28.9 

50.3 

30.7 

55.8 

31.4 

FS-3 

43 

1.4 

2.5 

2.7 

26.4 

T/O 

M/O 

T/O 

M/O 

FU-3 

43 

1.3 

2.5 

2.7 

26.8 

959.3 

235.3 

966.8 

241.2 

FS-4 

44 

2.1 

9.8 

4.63 

96.7 

T/O 

M/O 

T/O 

M/O 

FU-4 

44 

1.6 

2.5 

4.93 

119.8 

T/O 

M/O 

T/O 

M/O 

FS-5 

45 

7.7 

167.2 

16.69 

469.6 

T/O 

M/O 

T/O 

M/O 

FU-5 

45 

7.7 

153.9 

17 

506.5 

T/O 

M/O 

T/O 

M/O 
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Abstract.  In  this  paper,  we  present  the  first  steps  toward  a  runtime 
verification  framework  for  monitoring  hybrid  and  cyber-physical  sys¬ 
tems  (CPS)  development  tools  based  on  randomized  differential  test¬ 
ing.  The  development  tools  include  hybrid  systems  reachability  analysis 
tools,  model-based  development  environments  like  Simulink/Stateflow 
(SLSF),  etc.  First,  hybrid  automaton  models  are  randomly  generated. 

Next,  these  hybrid  automaton  models  are  translated  to  a  number  of 
different  tools  (currently,  SpaceEx,  dReach,  Flow*,  HyCreate,  and  the 
MathWorks’  Simulink/Stateflow)  using  the  HyST  source  transformation 
and  translation  tool.  Then,  the  hybrid  automaton  models  are  executed 
in  the  different  tools  and  their  outputs  are  parsed.  The  final  step  is  the 
differential  comparison:  the  outputs  of  the  different  tools  are  compared. 

If  the  results  do  not  agree  (in  the  sense  that  an  analysis  or  verification 
result  from  one  tool  does  not  match  that  of  another  tool,  ignoring  time¬ 
outs,  etc.),  a  candidate  bug  is  flagged  and  the  model  is  saved  for  future 
analysis  by  the  user.  The  process  then  repeats  and  the  monitoring  contin¬ 
ues  until  the  user  terminates  the  process.  We  present  preliminary  results 
that  have  been  useful  in  identifying  a  few  bugs  in  the  analysis  methods 
of  different  development  tools,  and  in  an  earlier  version  of  HyST. 

1  Introduction 

Runtime  verification  is  an  approach  to  ensure  the  correctness  and  reliability  of 
a  system  during  its  execution.  It  can  check  and  analyze  executions  of  a  sys¬ 
tem  under  scrutiny  that  violate  or  satisfy  a  given  correctness  property  by  us¬ 
ing  a  decision  procedure  called  a  monitor.  A  monitor  can  also  be  considered 
as  a  device  that  can  read  finite  traces  and  output  a  truth  value  derived  from 
a  truth  domain  [3].  Runtime  verification  can  be  used  broadly  for  many  pur¬ 
poses  such  as  debugging,  testing,  verification,  validation,  fault  protection,  and 
online  system  repair.  In  this  paper,  we  describe  a  preliminary  work  toward  a 
randomized  differential  testing  framework  [5]  that  may  be  used  as  a  runtime 
monitor  for  various  components  (from  parsers  to  analysis  algorithms)  in  hybrid 
and  CPS  analysis  tools  such  as  SpaceEx,  dReach,  Flow*,  HyCreate  and  the 
Mathworks’  Simulink/Stateflow  (SLSF).  A  test  subject  is  the  hybrid  automa¬ 
ton  randomly  generated  in  the  input  format  for  SpaceEx  using  a  prototype  tool 
called  HyRG  [4] 4 ,  which  is  then  translated  to  other  formats  including  dReach, 

4  The  tool  and  examples  are  available  online:  http://www.verivital.com/hyrg/ 
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Fig.  1:  Overview  of  monitoring  framework  for  hybrid  systems  analysis  tools  with 
randomized  differential  testing. 

Flow*,  HyCreate  and  SLSF  using  the  HyST  model  transformation  tool  [1].  Our 
contributions  include  (a)  the  first  steps  toward  a  randomized  differential  testing 
framework  to  monitor  CPS  development  and  verification  tools,  and  (b)  identify¬ 
ing  some  bugs  in  existing  tools,  including  a  semantic  difference  between  SpaceEx 
and  SLSF  that  we  did  not  know  about  and  some  soundness  bugs  in  the  verifica¬ 
tion  tools  that  have  been  corrected  by  the  tool  authors  [1]. 

2  Monitoring  with  Randomized  Differential  Testing 

We  first  describe  how  the  hybrid  systems  are  randomly  generated  in  HyRG 
so  they  have  diverse  continuous  and  discrete  behaviors.  We  then  analyze  these 
examples  with  different  hybrid  systems  development  and  verification  tools,  and 
then  compare  their  outputs  to  identify  possible  bugs  in  the  tools.  Figure  1  shows 
the  overview  of  our  framework  for  randomized  differential  testing  to  monitor 
hybrid  systems  development  tools.  First,  a  hybrid  automaton  Ar  is  randomly 
generated  by  HyRG,  then  Ar  is  translated  using  HyST  to  equivalent  automata 
in  different  tools’  formats,  denoted  A$,  Ar,  Ad,  Ah,  Am,  and  Ao-  Next,  the 
automata  can  be  analyzed  using  the  different  tools,  such  as  SpaceEx,  Flow*, 
dReach,  and  HyCreate,  or  simulated  in  SLSF.  Then  we  compare  all  analysis 
results  by  using  a  function  reachCheck  shown  in  Figure  2. 

The  reachCheck  function  has  three  inputs:  Reach,  Trace,  and  /3,  where  (3  is 
the  reachability  analysis  and  simulation  time  bound.  Reach  is  a  list  of  sets  of 
time-bounded  reachable  states  computed  by  different  tools  (e.g.,  the  output  of 
SpaceEx,  Flow*,  etc.).  Each  set  of  reachable  states,  7 Z(t),  is  the  set  of  states  that 
may  be  visited  by  following  the  model’s  trajectories  and  transitions,  for  any  time 
t  £  [0,  /?] .  That  is,  for  a  given  time  t ,  1Z(t)  is  the  set  of  states  reachable  at  time  t 
(sometimes  referred  to  as  a  time-slice).  The  input  Trace  is  a  set  of  all  simulation 
traces  produced  by  SLSF  up  to  a  maximum  simulation  time  /3. 
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1  function  reachCheck(Reach,  Trace, /3) 

foreach  set  of  reachable  states  7 Zi  in  Reach 
3  foreach  set  of  reachable  states  7 Zj  in  Reach 

if  i  7^  j  and  V£  £  [0,  /3]  7?-i(f)  A  IZj  (t)  is  UNSAT  then  return  UNSAT 
5  foreach  execution  trace  Tfc  in  Trace 

if  Vi  £  [0,  /3]  7fc  (i)  A  IZi  (t)  is  UNSAT  then  return  UNSAT 
7  return  SAT 

Fig.  2:  reachCheck  checks  whether  the  set  of  reachable  states  and  traces  computed 
by  different  tools  overlap  (have  non-empty  intersection)  at  every  time  instant. 

The  reachCheck  function  can  check  whether  the  reachable  states  or  simulation 
traces  computed  by  different  tools  at  each  time  have  non-empty  intersections. 
Although  all  of  the  reachable  states  and  simulation  traces  are  described  in  dif¬ 
ferent  formats  such  as  support  functions,  Satisfiability  Modulo  Theories  (SMT) 
formulas,  convex  sets,  etc.,  there  still  exists  an  equivalence  among  them.  For 
example,  reachable  sets  computed  by  SpaceEx’s  LGG  algorithm  are  a  represen¬ 
tation  of  convex  sets  (support  functions),  but  these  could  be  compared  to  the 
Taylor  models  of  Flow*.  If  the  reachable  sets  computed  by  different  tools  have  a 
non-empty  intersection  (pairwise  over  all  the  tools),  then  reachCheck  will  return 
SAT,  and  the  monitoring  continues  by  generating  a  different  random  model. 
Otherwise,  there  is  possibly  a  bug  in  the  HyST  translation  or  the  verification 
tools.  For  the  simulation  traces,  if  some  portions  of  a  trace  are  not  contained  in 
any  of  the  reachable  states,  reachCheck  will  return  UNSAT  and  there  is  again 
possibly  a  bug  in  HyST,  the  verification  tools,  or  SLSF.  Obviously  all  these  tools 
have  numerous  parameters,  so  numerical  issues,  accuracies,  etc.  must  be  taken 
into  account  by  the  user  to  determine  whether  a  candidate  bug  is  real. 

Next,  we  define  the  structure  of  a  hybrid  automaton  [2]  and  then  summarize 
the  random  generation  framework. 

Definition  1.  A  hybrid  automaton  TL  is  a  tuple,  TL  =  (Loc,  Var,  Flow,  Inv, 
Trans,  Init),  consisting  of  following  components:  (a)  Loc:  a  finite  set  of  dis¬ 
crete  locations,  (b)  Var:  a  finite  set  of  n  continuous,  real-valued  variables,  where 
\/ x  £  Var,  v(x)  £  TZ  and  v(x)  is  a  valuation — a  function  mapping  x  to  a 
point  in  its  type — here,  TZ;  and  Q  =  Loc  x  7 Zn  is  the  state  space,  (c)  Inv: 
a  finite  set  of  invariants  for  each  discrete  location,  VZ  £  Loc,  Inv(Z)  C  7 Zn . 

(d)  Flow:  a  finite  set  of  derivatives  for  each  continuous  variable  x  £  Var,  and 
Flow (l,x)  C  7 Zn  that  describes  the  continuous  dynamics  in  each  location  l  £  Loc. 

(e)  Trans:  a  finite  set  of  transitions  between  locations;  each  transition  is  a  tuple 
t  =  (src,  dst,  Grd,  Rst),  which  can  be  taken  from  source  location  src  to  destina¬ 
tion  location  dst  when  a  guard  condition  Grd  is  satisfied,  and  a  state  is  updated 
by  an  update  map  Rst.  (f)  Init:  an  initial  condition,  Init  C  Q. 

We  denote  a  hybrid  automaton  that  has  been  randomly  generated  by  Ar.  We 
randomly  generate  each  syntactic  component  of  the  automaton  Ar.  Rather  than 
picking  only  random  matrices  and  vectors  for  the  affine  functions  used  in  flows, 
guards,  invariants,  assignments,  etc.,  we  instead  partition  these  affine  functions 
into  classes.  While  we  assume  affine  functions  making  up  the  automaton,  the 
general  method  may  be  extended  to  nonlinear  functions.  We  highlight  that  all 
structural  components  of  the  automaton  are  selected  randomly  (i.e. ,  the  tran- 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

174 


4 


t  >  9 

t  0  A  X\  :=  ii  +  9  A  i2  :=  X2  +  15 


Fig.  3:  An  example  hybrid  automaton  Ar  with  time-dependent  switching  that 
was  randomly  generated  using  HyRG. 

sitions  and  continuous  dynamics),  and  are  not  simply  parameters.  For  brevity, 
we  do  not  describe  in  detail  the  random  generation  of  all  structural  components 
here,  but  refer  to  our  other  preliminary  results  [4]. 

3  Preliminary  Experimental  Results 

We  evaluate  our  preliminary3  monitoring  framework  in  several  scenarios  to 
compare  differences  among  several  hybrid  systems  verification  tools  including 
SpaceEx,  dReach,  and  Flow*,  as  well  as  SLSF  simulation.  Consider  a  randomly 
generated  hybrid  automaton  Ar  shown  in  Figure  3.  The  initial  state  of  Ar  is 
L0C3,  and  the  randomly  generated  initial  values  of  its  variables  are  respectively 
x\  =  10,  X2  =  17,  and  t  =  0.  Note  that  AR  is  nondeterministic.  The  results  of 
simulations  and  reachability  analysis  on  Ar  are  shown  in  Figure  4.  The  reachable 
states  restricted  to  x±  and  22  computed  by  Flow*  as  well  as  the  STC  and  LGG 
algorithms  in  SpaceEx  do  not  contain  a  simulation  trace  for  a  supposedly  equiv¬ 
alent  SLSF  model  created  using  HyST  when  Ar  takes  a  transition.  In  this  case, 
the  reachCheck  function  in  Figure  2  will  return  UNSAT.  This  happens  because 
of  semantic  differences  in  resets  among  Flow*,  SpaceEx,  and  SLSF.  In  SLSF,  the 
variables  x\  and  22  are  updated  sequentially,  so  that  x\  will  first  be  updated  to  a 
new  value,  and  then  22  will  be  updated  using  the  new  (already  updated)  value  of 
21 .  However,  these  variables  are  updated  concurrently  in  Flow*  and  SpaceEx  [2] , 
so  22  will  be  updated  by  using  the  previous  value  of  21.  Based  on  this,  we  fixed 
this  translation  error  in  HyST. 

4  Conclusion  and  Future  Work 

In  this  paper,  we  describe  our  preliminary  results  toward  building  a  randomized 
differential  testing  framework  to  monitor  hybrid  and  CPS  development  tools  like 
SLSF  and  verification  tools  like  SpaceEx,  dReach,  Flow*,  etc.  Our  preliminary 
results  include  identifying  semantic  mismatches  between  tools  automatically  that 
have  been  integrated  into  subsequent  versions  of  HyST.  Additionally,  we  have 
found  a  couple  bugs  in  some  of  the  verification  tools  that  have  been  corrected  by 

5  Some  of  the  steps  are  currently  manual,  particularly  the  parsing  of  reachable  states 
and  comparison  thereof,  but  the  generation  with  HyRG  and  translation  with  HyST 
is  fully  automatic. 
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(a)  (b) 

Fig.  4:  SLSF  simulation  (blue),  reachable  states  computed  by  Flow*  (green), 
SpaceEx’s  STC  algorithm  (red),  and  SpaceEx’s  LGG  algorithm  (gray)  for  AR 
showing  x\  and  X2  versus  time,  respectively.  The  SLSF  simulation  traces  and 
the  reachable  states  computed  by  Flow*,  SpaceEx’s  LGG  and  STC  algorithms 
do  not  line  up  (i.e. ,  have  an  empty  intersection)  at  some  points  in  time  (so 
reachCheck  returns  UNSAT)  due  to  a  semantic  difference. 

the  tool  authors.  Based  on  our  promising  preliminary  results,  we  plan  to  fully 
automate  every  step  of  the  framework  in  the  future. 
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ABSTRACT 

We  present  a  technique  to  investigate  abnormal  behaviors  of 
signals  in  both  time  and  frequency  domains  using  an  exten¬ 
sion  of  time-frequency  logic  that  uses  the  continuous  wavelet 
transform.  Abnormal  signal  behaviors  such  as  unexpected 
oscillations,  called  hunting  behavior,  can  be  challenging  to 
capture  in  the  time  domain;  however,  these  behaviors  can 
be  naturally  captured  in  the  time-frequency  domain.  We  in¬ 
troduce  the  concept  of  parametric  time-frequency  logic  and 
propose  a  parameter  synthesis  approach  that  can  be  used  to 
classify  hunting  behavior.  We  perform  a  comparative  anal¬ 
ysis  between  the  proposed  algorithm,  an  approach  based  on 
support  vector  machines  using  linear  classification,  and  a 
method  that  infers  a  signal  temporal  logic  formula  as  a  data 
classifier.  We  present  experimental  results  based  on  data 
from  a  hydrogen  fuel  cell  vehicle  application  and  electro¬ 
cardiogram  data  extracted  from  the  MIT-BIH  Arrhythmia 
Database. 

1.  INTRODUCTION 

For  the  last  decade,  signal  temporal  logic  (STL)  [11]  has 
been  successfully  extended  and  applied  in  many  domains 
such  as  exploring  requirements  for  closed-loop  control  sys¬ 
tems  [8],  identifying  oscillatory  behaviors  of  biology  sys¬ 
tems  [5] ,  and  formalizing  and  recognizing  music  melodies  [7] . 
Recently,  Kapinski  et  al.  introduced  a  new  signal  library 
template  for  constructing  formal  requirements  of  automo¬ 
tive  control  applications  using  STL  [10].  These  require¬ 
ments  involve  various  control  signal  behaviors  such  as  set¬ 
tling  time,  overshoot,  and  steady  state  errors.  Although 
most  of  such  control  signal  behaviors  can  be  characterized 
in  the  time  domain,  some  abnormal  signal  behaviors  such 
as  hunting  (undesirable  oscillations)  or  spikes  (abrupt,  mo¬ 
mentary  jumps  in  signal  values)  are  challenging  to  capture 
without  frequency  information.  In  most  practical  control 
systems,  hunting  behaviors  are  considered  undesirable,  or 
at  least  not  ideal,  and  care  is  taken  to  minimize  or  eliminate 
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the  behavior.  In  signal  processing,  hunting  behavior  can 
manifest  around  sharp  transitions,  as  a  result  of  compres¬ 
sion  artifacts;  this  occurs,  for  example,  in  image  processing, 
resulting  in  ghostly  bands  near  edges,  or  in  audio  compres¬ 
sion,  resulting  in  forward  echo  problems.  In  circuit  design,  a 
hunting  behavior  can  be  the  unwanted  oscillation  of  an  out¬ 
put  current  or  voltage,  which  may  cause  a  significant  rise 
in  power  consumption,  temperature,  electromagnetic  radia¬ 
tion,  or  settling  time  [9].  Although  some  hunting  behaviors 
can  be  defined  loosely  as  an  oscillation  around  a  given  av¬ 
erage  and  can  be  well  captured  using  STL,  some  modulated 
hunting  signals  are  challenging  to  detect  using  only  time 
domain  information  [10].  Because  hunting  signals  relate  to 
oscillatory  properties,  it  is  appropriate  to  investigate  them 
using  time-frequency  analysis. 

The  first  attempt  to  introduce  a  specification  formalism 
for  both  time  and  frequency  properties  of  a  signal,  called 
time- frequency  logic  (TFL),  was  proposed  by  Donze  and 
his  collaborators  [7].  There,  a  signal  is  preprocessed  using 
a  Short-Time  Fourier  Transform  (STFT)  [4]  to  generate  a 
spectral  signal  that  represents  the  evolution  of  the  STFT 
coefficients  at  some  particular  frequency  over  time.  The 
time-frequency  predicates  and  arithmetic  expressions  con¬ 
structed  from  this  spectral  signal  are  added  into  an  STL 
formula  to  yield  a  TFL  formula.  TFL  was  originally  applied 
to  music,  though  it  can  be  easily  extended  to  other  appli¬ 
cation  domains.  A  key  limitation  of  the  approach  using  the 
STFT  is  the  inherent  trade-off  required  between  resolution 
in  the  time  domain  and  resolution  in  the  frequency  domain; 
it  is  difficult  or  impossible  to  obtain  satisfactory  resolution 
in  both  time  and  frequency  using  the  STFT  for  the  analy¬ 
sis.  Such  limitations  can  be  overcome  using  the  continuous 
wavelet  transform  (CWT). 

In  the  following,  we  extend  the  notion  of  TFL  by  using 
the  CWT  to  specify  and  check  time-frequency  properties 
of  signals.  We  introduce  the  concept  of  parametric  time- 
frequency  logic  (PTFL)  and  use  it  to  perform  parameter 
synthesis  for  the  purpose  of  classifying  hunting  behavior. 
Previous  efforts  have  focused  on  data  classification  of  time- 
series  signals  using  STL  [2,3,8],  but  identifying  some  ab¬ 
normal  behaviors  such  as  hunting  requires  both  time  and 
frequency  information  [10].  Moreover,  existing  classification 
methods  require  an  extensive  amount  of  data,  and  the  in¬ 
ferred  classifier  is  often  difficult  for  engineers  to  interpret.  In 
contrast,  our  proposed  method  using  PTFL  can  efficiently 
classify  abnormal  behaviors  with  an  interpretable  data  clas¬ 
sifier  and  requires  less  data  than  existing  techniques.  We 
note  that  although  the  below  presentation  is  focused  on  one 
behavior  type,  it  is  straightforward  to  extend  the  work  to 
detect  other  abnormal  behaviors  such  as  noise,  spikes,  or 
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other  anomalous  behavior,  in  the  time-frequency  domain. 
We  evaluate  the  proposed  algorithm  by  comparing  the  per¬ 
formance  against  two  existing  classification  techniques:  a 
traditional  machine  learning  technique  using  a  support  vec¬ 
tor  machine  with  a  linear  kernel,  and  a  method  that  infers 
STL  formulae  as  data  classifiers  [3].  To  perform  the  eval¬ 
uation,  we  use  data  sets  from  two  different  domains,  the 
automotive  and  medical  domains. 

2.  TIME-FREQUENCY  LOGIC  USING  CWT 

Although  many  control  system  behaviors  can  be  naturally 
characterized  in  the  time  domain,  there  are  some  signal  be¬ 
haviors,  such  as  hunting  and  spikes,  that  are  challenging  to 
capture  without  frequency  information.  This  is  especially 
true  for  non-stationary  signals  whose  frequency  components 
vary  over  time;  for  this  class  of  signals,  it  is  essential  to  an¬ 
alyze  the  signal  properties  in  the  time-frequency  domain. 
STFT  is  a  popular  transformation  that  has  been  widely 
used  in  time-frequency  analysis  [4] .  Using  STFT  to  perform 
time-frequency  analysis,  a  signal  is  partitioned  into  small 
segments  (each  segment  is  assumed  to  be  stationary)  whose 
lengths  are  equal  to  the  width  of  a  chosen  window  function. 
The  window  function  is  used  to  modulate  the  signal  to  em¬ 
phasize  the  time  instant  associated  with  each  segment.  Un¬ 
fortunately,  the  STSF  provides  a  fixed  time-frequency  res¬ 
olution  so  that  it  is  not  effective  for  signals  that  need  to 
be  analyzed  with  different  time-frequency  resolutions  [14]. 
Moreover,  it  is  difficult  to  choose  a  proper  window  function 
with  an  appropriate  size  that  not  only  provides  both  desir¬ 
able  time  and  frequency  resolutions  but  also  does  not  violate 
the  stationarity  condition  [14].  To  overcome  the  limitation 
of  the  STFT,  we  use  the  CWT  to  analyze  a  signal  in  the 
time-frequency  domain. 

2.1  Continuous  Wavelet  Transform 

The  CWT  of  a  signal  x{t)  is  formally  defined  as  follows: 

/+oo 
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where  is  the  complex  conjugation  of  a  basic  wavelet 

function  which  is  derived  from  a  mother- wavelet  func¬ 

tion  ip(t).  This  function  has  zero  average  in  the  time  domain, 
i.e.  i/j(t)dt  =  0.  Furthermore,  a  basic  wavelet  function 

i pr  T(t)  can  be  written  as: 

(2) 

where  £  G  R>o  is  a  scale  parameter  representing  the  width  of 
the  basic  wavelet  function,  r  £  M  is  a  translation  factor  rep¬ 
resenting  the  location  of  the  basic  wavelet  function,  and 
is  the  energy  normalization  across  different  scales.  Thus,  the 
CWT  maps  an  original  signal  to  a  function  of  £  and  r  that 
provides  both  time  and  frequency  information.  Note  that 
the  scale  factor  is  inversely  proportional  to  the  frequency  of 
a  signal  [14] .  The  CWT  in  Equation  1  measures  the  similar¬ 
ity  between  a  basic  wavelet  function  and  a  signal.  Indeed, 
if  a  signal  x(t )  has  a  frequency  component  /  corresponding 
to  a  particular  scale  (  of  a  wavelet  function  ip^tT(t),  then 
the  portion  of  x{t)  at  some  particular  time  interval  where  / 
exists  will  be  similar  to  ipc,r(t)-  As  a  result,  the  CWT  co¬ 
efficients  of  x(t)  corresponding  to  /  will  be  relatively  large 
over  this  time  interval.  Moreover,  the  time-frequency  energy 
density  of  the  CWT  is  equivalent  to  the  square  norm  of  the 
CWT  coefficients: 

JV/(C,t)  =  |W/(C,t)|2.  (3) 


Time-frequency  resolution.  In  contrast  to  the  STFT, 

the  CWT  can  either  dilate  or  compress  the  window  size  of 
the  wavelet  function,  and  translate  it  along  the  time  axis. 
The  Heisenberg  box  [12]  is  a  range  of  times  and  frequencies 
that  indicates  the  accuracy  of  a  time-frequency  transforma¬ 
tion.  Although  the  area  of  the  Heisenberg  box  does  not 
change,  the  time  and  frequency  resolutions  can  be  varied 
depending  on  the  value  of  £.  As  a  result,  the  CWT  can 
analyze  all  frequency  components  within  a  signal  by  con¬ 
sidering  appropriate  scales  of  the  mother-wavelet  function. 
For  instance,  the  CWT  can  use  the  wavelet  function  with 
a  short  duration  and  low  scale  for  analyzing  high  frequency 
components,  and  vice  versa.  This  advantage  of  the  CWT  al¬ 
lows  us  to  efficiently  analyze  a  signal  that  includes  abnormal 
behaviors  such  as  spikes  and  hunting. 

2.2  Time-Frequency  Logic 

TFL  is  an  extension  of  STL  that  can  be  used  to  specify 
both  time  and  frequency  properties  of  a  signal  [7].  In  TFL, 
a  signal  predicate  is  defined  over  the  signal  representing  the 
evolution  of  the  STFT  coefficient  at  a  particular  frequency 
over  time.  Given  a  pair  (/,  t)  of  frequency  and  time,  the 
STFT  of  a  signal  x(t)  is  obtained  by: 

/+oo 
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where  V'n(f)  is  a  window  function.  A  spectral  signal  y(t )  = 
|S/,t|2  is  the  projection  of  the  spectrogram  of  x(t)  on  a  par¬ 
ticular  frequency  /.  Such  a  signal  can  be  incorporated  in 
TFL  formulae  to  form  some  interesting  time-frequency  spec¬ 
ifications.  We  can  see  that  a  TFL  formula  is  actually  an  STL 
formula  in  which  the  signal  predicate  is  defined  over  y(t)  in¬ 
stead  of  x(t).  TFL  has  been  used  to  formalize  and  recognize 
music  melodies,  where  time-frequency  requirements  are  sim¬ 
ply  specified  as  tp  =  |S/p,t|2  >  9 ,  where  fp  is  the  pitch  fre¬ 
quency  and  6  is  the  STFT  coefficient  threshold  [7];  however, 
the  shortcomings  of  the  STFT  mentioned  previously  may 
reduce  the  ability  of  TFL  to  precisely  specify  and  evaluate 
time-frequency  properties  of  a  signal.  We  extend  TFL  to 
use  the  CWT  to  obtain  spectral  signals  from  a  given  time- 
series  signal.  In  effect,  we  construct  a  TFL  formula  based 
on  the  CWT  coefficients  of  the  spectral  signals  instead  of 
the  STFT  coefficients.  Because  the  CWT  can  appropriately 
use  various  scaling  factors,  £,  to  analyze  all  frequency  com¬ 
ponents  at  different  time  intervals,  it  gives  us  an  ability  to 
study  signals  at  flexible  time-frequency  resolutions. 

Although  the  following  presentation  focuses  on  the  clas¬ 
sification  of  hunting  behaviors,  we  note  that  the  proposed 
approach  using  TFL  and  CWT  can  be  used  to  capture  other 
time-frequency  specifications  as  well.  For  instance,  consider 
the  property:  “ For  some  time  in  the  future,  the  dominant 
frequency  of  the  signal  is  lo  for  5  time  units,  and  the  domi¬ 
nant  frequency  subsequently  rises  to  twice  of  this  value  within 
10-time  units.'’  Here,  the  dominant  frequency,  f(t),  of  a  sig¬ 
nal  x(t)  is  defined  as  the  frequency  corresponding  to  the 
maximum  magnitude  frequency  component  of  the  signal  at 
time  t,  as  provided  by  a  CWT.  Such  a  time-frequency  prop¬ 
erty  can  be  written  as  a  TFL  formula,  =  0(C[o,5](/  = 
oi)  A 0[5,i5] (/  =  2 tu)).  Then,  the  TFL  formula  ip  can  be  eval¬ 
uated  as  a  normal  STL  formula  using  Breach1  [6] .  Consider 
another  property  such  as  “At  some  time  in  the  future  the 
energy  densities  of  the  signal  within  a  particular  time  inter- 

1  Breach  [6]  is  a  tool  that  allows  evaluation  of  STL  and  TFL 

formulae  on  signals. 
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val  and  a  particular  frequency  bandwidth  are  always  greater 
than  some  threshold  value  9.”  This  property  can  be  specified 
as  a  TFL  formula,  <f>  =  0D[t1,*2]  (-*(/,  t)  >  6),  where  z(f,t) 
is  a  spectral  signal  that  captures  the  minimum  value  of  the 
CWT  coefficients  of  a  signal  over  some  frequency  bandwidth 
U1J2]. 

Parametric  Time-Frequency  Logic.  We  introduce  para¬ 
metric  time-frequency  logic  (PTFL),  which  is  an  extension 
of  TFL  where  the  parameters  in  TFL  template  formulae  are 
symbolic  parameters.  Similar  to  the  concept  of  parameter 
signal  temporal  logic  (PSTL)  introduced  in  [1] ,  PTFL  allows 
constants  in  intervals  bounding  the  temporal  operators  and 
constant  values  in  the  predicates  of  PTFL  formulae  to  be 
replaced  with  parameters. 

The  p  parameters  in  a  PTFL  formula  are  classified  into 
two  sets: 

(a)  T  =  {n, ...,  rpt }  is  a  set  of  pt  time  parameters  occurring 
in  the  time  intervals  of  the  temporal  operators,  and 

(b)  6  =  {9i,  •  9p-pt}  is  a  set  of  p  —  pt  threshold  parame¬ 
ters  occurring  in  the  signal  predicates. 

For  any  fixed  values  of  T  and  0,  a  PTFL  formula  r*p(j\ , . . . ,  rpi , 
9\, . . . ,  9P-Pt )  yields  a  TFL  formula  corresponding  to  the 
fixed  values  of  the  parameters.  For  instance,  consider  a 
PTFL  formula  <p(r,9 )  =  □[o,T](j/(t)  >  9),  where  y(t)  is  a 
spectral  signal,  r  and  9  are  time  and  threshold  parameters, 
respectively.  The  formula  <p(5, 10)  is  defined  as  the  TFL 
formula  □[0,5] {y(t)  >  10). 
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Figure  1:  A  sketch  illustrates  the  hunting  classifica¬ 
tion  problem  using  time-frequency  parameter  syn¬ 
thesis.  The  set  of  spectral  signals  W f,  is  acquired 
from  the  CWT  of  an  original  time-series  signal. 

□  a  set  of  labeled  traces  'F  =  {’Fa,  'F/3},  where  and 
'F/3  denote  a  set  of  training  and  testing  traces,  respec¬ 
tively.  Moreover,  we  the  notation  'F.B  and  fF.G  to 
respectively  denote  the  set  of  traces  with  and  without 
hunting  behavior.  Note  that  all  traces  in  the  training 
set  exhibit  hunting  behavior,  so  that  'FQ  =  'Fa.-B 

□  a  cut-off  frequency  5. 

□  sets  of  parameters  T,  and  0. 

•  Find  values  for  T  and  0,  such  that: 

□  Xj(t )  |=  ifih( T,0)  for  all  Xj(t)  G  'I’p.B. 

□  Xj(t )  |^=  ifh( T,0)  for  all  Xj(t)  G  'S  p .G. 


3.  HUNTING  CLASSIFICATION 

In  this  section,  we  will  describe  three  different  approaches 
using  PTFL  and  TFL  to  efficiently  classify  hunting  behav¬ 
iors  in  signals.  Informally,  a  hunting  behavior  is  an  unde¬ 
sirable  oscillation  appearing  within  a  signal  over  some  time 
interval. 

3.1  Parameter  Synthesis  Approach 

We  now  propose  a  method  to  classify  hunting  behavior 
based  on  mining  parameters  of  the  following  PTFL  formula: 

m 

<Ph=  A0[0,T«](W7i(t)  >9i).  (5) 

Intuitively,  this  formula  specifies  that  “ the  energy  densities 
of  the  given  signal  at  particular  frequencies  are  eventually 
greater  than  some  threshold  value”.  Here,  W fi(t)  is  a  spec¬ 
tral  signal  over  time  that  captures  the  energy  densities  of  the 
CWT  of  an  original  time-series  signal  x(t)  at  a  particular  fre¬ 
quency  fi  G  F.  Note  that  F  is  a  set  of  frequencies  based  on 
the  scales  of  the  CWT.  Each  spectral  signal,  W fi(t),  is  the 
row  vector  of  the  matrix  representing  the  energy  densities 
of  the  CWT  of  x(t);  such  a  matrix  is  obtained  using  Equa¬ 
tion  1  and  Equation  3.  Also,  n  G  T  and  9i  G  0  denote  a 
time  and  threshold  parameter  corresponding  to  each  spec¬ 
tral  signal  W fi(t).  We  note  that  the  satisfaction  value  of 
the  property  tfih  monotonically  increases  in  n  and  decreases 
in  6i.  Because  of  monotonicity,  we  can  exponentially  reduce 
the  search  over  the  parameter  space  so  that  the  synthesis 
procedure  is  efficient  [8].  Figure  1  conceptually  illustrates  a 
spectral  signal  Wfi(t),  and  an  instance  of  a  hunting  behav¬ 
ior  that  may  occur  within  a  signal.  We  say  that  a  signal  x(t) 
contains  hunting  behavior  if  the  property  tph  holds.  Overall, 
the  hunting  classification  problem  can  be  written  as  follows. 

•  Given  the  following  inputs: 


We  introduce  the  cut-off  frequency  <5  to  reduce  the  effort  to 
exhaustively  mine  parameters  over  the  entire  time- frequency 
domain.  It  is  essential  for  the  control  engineers  to  indicate 
that  hunting  behavior  only  occurs  at  some  high-frequency 
region  above  8. 

Classification  Algorithm.  Next,  we  propose  a  heuristic  to 
automatically  obtain  values  for  T  and  0  that  can  be  used  to 
separate  the  hunting  and  non-hunting  signals.  An  overview 
of  the  heuristic  is  described  in  Algorithm  1.  The  heuristic 
can  be  interpreted  as  follows. 

Line  2  initializes  a  matrix  E  that  represents  the  k  m- 
dimensional  spectral  signals  transformed  from  k  original  time- 
series  signals  in  the  training  set  using  the  CWT.  We  iterate 
over  each  trace  in  to  construct  sets  of  spectral  signals 
{W fi(t), ...,  W fm(t)}  using  the  CWT,  and  assign  them  to  E. 
Next,  we  call  the  function  TruncateParam  to  reduce  the  effort 
of  exhaustively  mining  all  parameters  over  the  entire  time- 
frequency  domain.  Here,  E'  represents  the  k  n-dimensional 
(n  <  m)  matrix  of  E  corresponding  to  the  frequency  range 
above  5.  Next,  we  call  the  function  HuntingParamSyn  incor¬ 
porated  inside  Breach  to  mine  values  for  T  and  0.  Then, 
we  test  the  classifier  with  a  given  set  of  testing  traces  'F/3. 
The  function  Classifier  checks  the  satisfaction  of  ifih  for  each 
trace  in  'S> p,  and  returns  the  misclassification  rate  (MCR) 
value  and  the  set  of  misclassified  traces  'Fm.  The  values  of 
T,  0  and  the  set  'Fm  are  then  returned  for  further  anal¬ 
ysis.  Furthermore,  we  can  call  EnhancedParam  function  to 
strengthen  the  values  T  and  0  and  reduce  the  MCR  value 
for  the  purpose  of  optimizing  the  classifier  formula.  Note 
that  in  the  case  studies,  we  do  not  use  this  function  to  eval¬ 
uate  the  performance  of  the  classifier  to  avoid  the  bias  in 
our  comparative  analysis. 

3.2  Decision  Tree  Approach 

An  approach  based  on  decision  trees  to  classify  time  se¬ 
ries  data  using  STL  formulae  was  implemented  in  the  tool 
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Algorithm  1  Hunting  Classification  Using  Parameter  Syn¬ 
thesis 

1  function  HuntingClassification('&a,  8) 

E  «-  0 

3  for  each  trace  Xj{t)  £  He,  j  <  k 

E(j, :)  «-  Wh{t), ...,  Wfm(t)  <-  CWT(Xj(t )) 

5  end  for 

S'  «—  TruncateParam(<5,  E) 

7  T,  0  <—  HuntingParamSyn(E') 

M C/7,  £-  Classifier(T,  0,  vEfi,) 

9  return  T,  0,  'I'm 

end  function 

11  function  EnhancedPararn('l/m,  8) 

if  >3 >m.B  A  0  then 
13 

HuntingClassification('I'((,  ,g ,  5) 

15  end  if 

end  function 


DT4STL  [3].  That  method  uses  a  parameterized  procedure 
to  infer  STL  formulae  from  labeled  data.  Given  a  two-class 
training  data  and  a  set  of  PSTL  templates,  a  decision  tree 
for  classification  is  recursively  built  such  that  each  node  of 
a  tree  is  associated  with  a  simple  formula,  selected  from  the 
given  PSTL  templates.  The  parameter  synthesis  is  then  con¬ 
ducted  to  find  the  STL  formula  that  yields  the  best  split  for 
the  data  at  each  node.  This  technique  can  be  used  to  auto¬ 
matically  construct  classifiers  based  on  STL  formula,  but  to 
achieve  a  low  MCR  value,  the  inferred  STL  formulae  may 
be  long  and  not  easily  interpretable  by  engineers.  In  this 
section,  we  apply  this  approach  to  classify  hunting  versus 
non- hunting  signals.  Instead  of  inferring  an  STL  formula, 
we  intend  to  infer  a  TFL  formula  as  a  data  classifier.  Thus, 
we  transform  original  time  series  data  into  a  collection  of 
time- frequency  data  (spectral  signals). 

We  assume  that  control  engineers  initially  designate  the 
frequency  threshold  separating  hunting  versus  non-hunting 
behavior.  A  hunting  behavior  is  specified  as  any  oscilla¬ 
tory  behavior  occurring  at  frequencies  above  some  specified 
cut-off  frequency  8.  Thus,  the  time- frequency  profile  of  a 
hunting  signal  at  some  frequency  component  f  >  8  con¬ 
tains  larger  values  for  the  CWT  coefficients  compared  to 
those  of  non-hunting  signals.  So  we  define  the  spectral  sig¬ 
nal  WThcoef  based  on  the  CWT  coefficients  of  the  signal  in 
a  high-frequency  region  such  that: 

WThcoef(t)  =  max  ,  Pwf(C,t),  (6) 

feir _ Lq _  fc  I 

h‘,„  ax  ■  Ts  5  ' 

where  fc  is  a  center  frequency  associated  with  the  mother- 
wavelet  function,  Pmol  is  the  maximum  frequency  that  ap¬ 
pears  in  the  CWT,  and  Ts  is  the  sampling  period.  We  use 
such  a  spectral  signal  as  an  input  for  the  DT4STL  to  infer  a 
simple  TFL  formula.  Note  that  in  this  scenario,  the  inferred 
TFL  formula  captures  the  non-hunting  behavior  of  a  signal. 

3.3  Support  Vector  Machine  Approach 

Next,  we  present  another  approach  that  can  solve  the 
problem  of  hunting  classification:  linear  classification  us¬ 
ing  support  vector  machines  (SVM)  [15].  A  linear  SVM  is 
a  set  of  hyperplanes  or  decision  boundaries  that  can  cor¬ 
rectly  separate  data  into  two  classes.  The  general  form  of 
hyperplanes  is  (w  ■  x)  +  b  =  0,  where  w  is  a  normal  to  the 
hyperplane,  and  is  the  perpendicular  distance  from  the 
hyperplane  to  the  origin.  The  sign  of  the  linear  discriminant 


function  f(x)  =  {w  ■  x)  +  b  determines  on  which  side  of  the 
decision  boundary  the  test  data  point  is  located.  The  dis¬ 
tance  from  the  decision  boundary  to  the  closest  data  point 
determines  the  margin  of  the  linear  classifier.  Suppose  that 
we  have  a  set  of  n  labeled  training  data  ( Xi ,  d), ...,  (xn,  c„) 
where  Xi  £  R“  and  d  £  {1,-1},  the  constrained  optimiza¬ 
tion  problem  of  linear  classification  using  SVM  is  written 
as: 

1 

minimize  - 1  Iwl  I2  +  C  C; 

w,b  2  1  ' 

i= 1 

subject  to  d ({w  ■  Xi)  +  6)  >  1  —  Q,  i  =  1, . . .  ,n 

C<  >  0.  (7) 

Here,  £  is  a  slack  variable.  If  0  <  £  <  1,  the  data  point  lies 
somewhere  between  the  margin  and  the  correct  side  of  liy- 
perplane,  and  the  data  point  is  misclassified  if  £  >  1.  C  is  a 
regularization  parameter  that  defines  the  trade-off  between 
errors  of  the  SVM  on  training  data  and  margin  maximiza¬ 
tion.  A  large  value  of  C  results  in  the  low  possibility  of 
misclassified  training  data  points,  because  the  optimization 
in  Equation  7  will  choose  a  narrow  margin  hyperplane  that 
correctly  separates  training  data  points  as  much  as  possi¬ 
ble.  In  contrast,  a  small  value  of  C  will  result  in  a  large 
margin  hyperplane,  but  it  may  yield  a  better  result  in  terms 
of  correctly  separating  testing  data  points.  Due  to  space 
limitation,  we  will  not  discuss  the  formal  optimization  prob¬ 
lem  solved  to  obtain  the  SVM,  but  refer  interested  readers 
to  [15].  In  this  work,  instead  of  applying  the  linear  SVM  di¬ 
rectly  to  original  time  series  signals,  we  need  to  preprocess 
them  to  yield  a  corresponding  set  of  time- frequency  features. 
For  each  time-series  signal  x{t),  we  collect  a  real- valued  vec¬ 
tor  Wmax  =  \Wfr*,..;Wfr*]  such  that  each  element 
Wfhax  £  maxjmum  value  of  a  spectral  sig¬ 

nal  Wfi(t).  Such  a  vector  will  be  used  as  a  time- frequency 
feature  to  design  the  SVM. 

4.  CASE  STUDIES 

In  this  section,  we  evaluate  the  capabilities  of  three  dif¬ 
ferent  methods  to  classify  hunting  behavior  for  two  case 
studies.  The  first  case  study  is  based  on  data  from  an  air 
compressor  motor  speed  (ACMS)  system  in  a  fuel  cell  (FC) 
vehicle  application.  The  second  case  study  is  based  on  elec¬ 
trocardiogram  (ECG)  data.  In  both  examples,  we  apply  the 
Morlet  CWT  [12]  to  the  time-series  signals. 

4.1  ACMS  Data 

The  ACMS  system  uses  a  compressor  to  regulate  the  air 
intake  of  a  hydrogen  FC  vehicle.  An  FC  stack  uses  a  mix¬ 
ture  of  air  and  hydrogen  to  generate  electrical  power  for  the 
vehicle.  Accurate  control  of  the  compressor  which  translates 
to  control  of  the  quantities  of  hydrogen  and  oxygen  (air)  is 
required  to  achieve  good  performance  and  proper  operation 
from  the  FC  stack.  Also,  the  water  balance  (moisture  level) 
within  the  stack  needs  to  be  carefully  regulated,  which  re¬ 
quires  regulation  of  the  air  pressure  at  the  inlet  of  the  stack. 
The  task  of  the  ACMS  system  is  to  regulate  air  flow  and  air 
pressure  delivered  to  the  inlet  of  the  FC  stack. 

We  consider  ACMS  data  from  an  FC  vehicle  application. 
Specifics  of  the  data,  such  as  units  and  descriptions  of  the 
measured  quantities  are  omitted  here  for  proprietary  rea¬ 
sons.  The  ACMS  data  are  partitioned  into  a  collection  of 
traces  that  are  100  seconds  in  length  and  are  labeled  as  ei¬ 
ther  good  (the  trace  does  not  exhibit  hunting  behavior)  or 
bad  (the  trace  does  exhibit  hunting  behavior).  The  ACMS 
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Figure  2:  The  classified  testing  data  of  the  ACMS 
signals  using  parameter  synthesis  approach. 

data  has  a  sampling  period  of  0.02  seconds.  We  note  that 
the  same  training  data  is  used  for  all  of  the  evaluations, 
though  the  parameter  synthesis  approach  only  uses  the  bad 
traces.  In  this  experiment,  we  use  the  training  data  includ¬ 
ing  50  total  traces,  in  which  30  traces  are  labeled  as  good 
and  the  others  are  labeled  as  bad.  We  also  use  the  same 
testing  data  including  10  good  traces  and  10  bad  traces  for 
all  of  the  evaluations. 

Parameter  Synthesis.  We  now  illustrate  the  performance 
of  the  classification  heuristic  shown  in  Algorithm  1  to  clas¬ 
sify  hunting  behavior  for  the  ACMS  signals.  Because  we 
do  not  know  the  frequency  range  where  a  hunting  behavior 
may  occur,  we  exhaustively  mine  all  parameters  n  £  T  and 
9i  £  0.  We  choose  the  maximum  frequency  of  the  CWT 
as  Fmax  =  25Hz.  Here,  the  Algorithm  1  will  search  for  the 
best  6i  £  [0, 1]  and  n  £  [0, 100]  such  that  all  spectral  signals 
transformed  from  original  time-series  traces  in  the  training 
data  satisfy  iph-  We  then  use  Breach  with  the  optimized 
parameters  of  iph  to  classify  good  versus  bad  traces  in  the 
testing  set. 

Figure  2  shows  the  experimental  results  of  classifying  ab¬ 
normal  ACMS  signals,  using  the  function  HuntingClassification 
In  the  figure,  we  only  show  five  representative  signals  in 
which  good  traces  correctly  classified  are  shown  in  green, 
and  bad  traces  correctly  classified  are  shown  in  blue.  The 
one  good  trace  that  is  misclassified  is  shown  in  red.  The  total 
running  time  of  the  classification  process  is  approximately  3 
minutes. 

Decision  Tree  Approach.  Next,  we  utilize  the  DT4STL 

toolbox  to  infer  TFL  formulae  that  can  be  used  to  classify 
hunting  behavior  for  the  ACMS  data. 

We  preprocess  the  training  data  to  yield  the  corresponding 
set  of  spectral  signals  WThcoef  with  S  =  15Hz  and  Fmax  = 
25Hz.  We  then  run  the  DT4STL  toolbox  with  this  set  of 
spectral  signals  using  2-fold  cross-validation.  As  a  result, 
we  obtain  the  two  following  TFL  formulae: 

tphi  =  □[37.4,98.2)  (WThcoef  <  0.0435) 

iph2  =  □[1.29,91.3) (WThcoef  <  0.0394). 

The  procedure  takes  approximately  75  seconds  to  infer  each 
formula.  Using  Breach,  we  then  evaluate  those  formulae 
with  the  set  of  testing  data.  The  formula  iphi  gives  us  all 
misclassified  traces  that  are  bad  traces  with  the  MCR  value 
being  equal  to  25%.  On  the  other  hand,  the  formula  iph2 
results  in  one  misclassified  trace,  which  is  a  bad  trace. 

SVM  Approach.  We  apply  the  SVM  method  to  classify 
normal  versus  abnormal  ACMS  data.  We  first  transform  all 
of  the  traces  in  the  training  data  into  sets  of  time-frequency 
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Figure  3:  The  classified  testing  data  of  the  ECG 
signals  using  parameter  synthesis  approach. 

features.  Next,  we  run  the  linear  SVM  to  learn  the  deci¬ 
sion  boundaries  that  separate  data  as  either  good  or  bad. 
Finally,  we  predict  the  testing  data  from  the  learned  deci¬ 
sion  boundaries  with  different  values  of  the  SVM  classifier 
margin  C. 

The  MCR  of  the  hunting  classification  for  the  ACMS  data 
using  SVM  is  10%  with  C  =  10  and  reduces  to  5%  with 
C  =  100.  In  this  case,  a  larger  value  of  C  gives  a  better  result 
for  the  classification.  Moreover,  the  classification  process 
takes  only  0.393  seconds. 

4.2  ECG  Data 

An  electrocardiogram  (ECG)  test  is  a  noninvasive  proce¬ 
dure  used  to  monitor  the  electrical  activities  of  a  heart  via 
a  collection  of  electrodes  attached  to  the  patient’s  skin.  A 
doctor  can  read  an  ECG  output  signal  to  diagnose  abnor¬ 
mal  structure  or  function  of  the  patient’s  heart.  A  normal 
ECG  signal  includes  three  signals:  (a)  the  P  wave  repre¬ 
senting  the  depolarization  or  contraction  of  the  atrium  (b) 
the  QRS  complex  (the  R  wave)  indicating  the  ventricular 
depolarization  and  (c)  the  T  wave  describing  the  ventricu¬ 
lar  repolarization.  The  distance  between  two  consecutive  R 
peaks  is  considered  as  a  heartbeat.  A  healthy  patient  has  a 
resting  normal  heartbeat  (frequency)  from  60  to  100  beats 
per  minute  (bpm). 

In  this  paper,  we  focus  on  classifying  the  ECG  signal 
that  may  contain  a  ventricular  tachycardia  (VT),  a  very 
fast  heart  rhythm  arising  in  the  ventricles  that  may  cause  a 
sudden  heart  failure.  VT  is  defined  as  a  sequence  of  three 
or  more  ventricular  beats  with  the  frequency  varying  from 
110  to  250  bpm.  Thus,  a  VT  can  be  considered  as  a  hunting 
behavior  in  an  ECG  signal.  We  conduct  our  classification 
approaches  on  the  MIT-BIH  Arrhythmia  ECG  Database. 
These  data  contain  a  variety  of  ECG  signals  collected  from 
patients  23  to  89  years  of  age,  including  patients  who  expe¬ 
rience  ventricular  arrhythmia  [13].  We  transform  ECG  sig¬ 
nals  20  seconds  in  duration  (provided  at  a  sampling  period 
of  0.0028  secs.)  to  spectral  signals  using  the  Morlet  CWT. 
Here,  the  maximum  frequency  of  the  CWT  is  Fmax  =  4.5Hz 
(~  270  bpm).  For  all  of  the  evaluations,  we  use  the  same 
training  data  including  20  bad  traces  (the  traces  do  contain 
a  VT)  and  40  good  traces  (the  traces  do  not  contain  a  VT), 
and  the  same  testing  data  including  10  good  traces  and  10 
bad  traces. 

Parameter  Synthesis.  In  this  scenario,  we  only  mine  the 
parameters  for  20  bad  traces  in  the  training  dataset.  Here, 
we  will  search  for  the  best  9i  £  [0,  5]  and  n  £  [0,  20].  Fig¬ 
ure  3  shows  the  experimental  results  of  using  the  function 
HuntingClassification  to  classify  abnormal  ECG  signals  that 
contain  VT.  Here,  we  only  show  three  signals  for  illustra¬ 
tion.  The  approach  results  in  one  (5%)  misclassified  (red) 
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PS 

DT4STL 

SVM 

Interpretation  of  data  classifier 

o 

A 

X 

Computation  time 

X 

X 

o 

Bad  behavior  localization 

o 

o 

X 

Low  misclassification  rate 

A 

A 

o 

Table  1:  The  comparison  between  parameter  syn¬ 
thesis  (PS)  using  PTFL,  DT4STL  toolbox  using  TFL, 
and  linear  SVM  in  classifying  abnormal  signals, 
where  O;  A>  x  respectively  denote  good,  ok,  bad. 

trace,  which  is  a  bad  trace.  The  total  running  time  of  the 
classification  process  is  approximately  1  minute. 

Decision  Tree  Approach.  Next,  we  utilize  the  DT4STL 

toolbox  to  classify  hunting  behavior  for  the  ECG  data.  We 
first  preprocess  the  training  data  to  yield  the  corresponding 
set  of  spectral  signals  WThcoef  with  S  =  1.5Hz.  Then,  we 
run  the  DT4STL  toolbox  with  this  set  of  spectral  signals 
using  2-fold  cross-validation.  As  a  result,  we  obtain  two 
following  TFL  formulae: 

(phi  =  □[i.73,i7.3)  (WThcoef  <  3.16) 

<Ph2  —  d[2. 36, 20)  (WThcoef  <  3.21). 

The  procedure  takes  approximately  105  seconds  to  infer  each 
formula.  We  then  use  Breach  to  evaluate  these  formulae  with 
a  set  of  spectral  data  acquired  from  the  CWT  of  10  good 
traces  and  10  bad  traces  in  the  testing  data.  The  MCR 
values  of  using  (phi  and  (phi  to  classify  these  data  are  both 
equal  to  5%  (but  misclassified  traces  are  different). 

SVM  Approach.  Finally,  we  apply  the  SVM  approach  to 
classify  hunting  in  the  ECG  data.  Note  that  we  use  the  same 
training  and  testing  data  used  for  the  other  methods.  The 
hunting  classification  of  the  ECG  data  using  an  SVM  results 
in  a  5%  MCR  for  all  values  of  C  (the  one  misclassified  trace 
is  a  bad  trace),  and  the  classification  procedure  takes  0.3 
seconds. 

5.  DISCUSSION 

In  this  section,  we  discuss  the  trade-offs  related  to  the 
three  classification  approaches  presented  above  to  classify 
normal  versus  abnormal  signals.  Table  1  shows  an  aggregate 
performance  evaluation  between  the  approaches  in  four  dif¬ 
ferent  categories,  including  (a)  the  ability  to  interpret  the 
structure  and  parameters  used  to  define  the  classifier,  (b) 
the  computation  time,  (c)  the  capacity  to  localize  where 
bad  behavior  occurs  in  a  signal,  and  (d)  the  ability  to  cor¬ 
rectly  classify  normal  versus  abnormal  signals.  Although  the 
linear  SVM  can  classify  abnormal  signals  much  faster  and 
more  accurately  than  the  parameter  synthesis  and  the  deci¬ 
sion  tree  approaches,  the  main  drawback  of  this  method  is 
that  it  cannot  reveal  where  the  bad  behavior  occurs  within 
a  signal.  We  found  that  the  decision  tree  approach  can  infer 
specifications  that  accurately  classify  data  as  either  good  or 
bad;  however,  it  is  not  easy  to  interpret  the  inferred  formula 
unless  the  user  has  some  expertise  about  the  input  data.  If 
a  dataset  is  not  homogeneous  (i.e.,  both  normal  and  abnor¬ 
mal  signals  are  very  different  from  each  other),  the  DT4STL 
toolbox  may  infer  a  complicated  formula  that  cannot  be  eas¬ 
ily  interpreted.  The  parameter  synthesis  using  PTFL  and 
the  decision  tree  approach  using  TFL  have  similar  perfor¬ 
mance  except  the  former  provides  a  clearer  intuition  about 
the  classifier,  as  the  temporal  logic  formula  that  results  is 
usually  simpler  for  the  PTFL  case.  Overall,  we  conclude 


that  a  traditional  machine  learning  technique  such  as  the 
linear  SVM  is  the  best  choice  if  the  only  goal  is  to  classify 
data  as  either  good  or  bad,  and  the  most  important  thing  is 
to  select  a  proper  feature  on  which  to  base  the  classification 
algorithm.  Otherwise,  if  the  designer  additionally  wishes  to 
both  understand  the  meaning  of  a  data  classifier  and  auto¬ 
matically  localize  where  abnormal  behaviors  occur  within  a 
signal,  we  conclude  that  the  parameter  synthesis  approach 
is  the  best  option,  as  a  simple  temporal  logic  formula  that 
defines  the  classifier  results  from  the  analysis. 
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ABSTRACT 

A  hyperproperty  is  a  property  that  requires  two  or  more  execution 
traces  to  check.  This  is  in  contrast  to  properties  expressed  using 
temporal  logics  such  as  LTL,  MTL  and  STL,  which  can  be  checked 
over  individual  traces.  Hyperproperties  are  important  as  they  are 
used  to  specify  critical  system  performance  objectives,  such  as 
those  related  to  security,  stochastic  (or  average)  performance,  and 
relationships  between  behaviors.  We  present  the  first  study  of  hy¬ 
perproperties  of  cyber-physical  systems  (CPSs).  We  introduce  a  new 
formalism  for  specifying  a  class  of  hyperproperties  defined  over 
real-valued  signals,  called  HyperSTL.  The  proposed  logic  extends 
signal  temporal  logic  (STL)  by  adding  existential  and  universal  trace 
quantifiers  into  STL’s  syntax  to  relate  multiple  execution  traces. 
Several  instances  of  hyperproperties  of  CPSs  including  stability, 
security,  and  safety  are  studied  and  expressed  in  terms  of  HyperSTL 
formulae.  Furthermore,  we  propose  a  testing  technique  that  allows 
us  to  check  or  falsify  hyperproperties  of  CPS  models.  We  present  a 
discussion  on  the  feasibility  of  falsifying  or  verifying  various  classes 
of  hyperproperties  for  CPSs.  We  extend  the  quantitative  semantics 
of  STL  to  HyperSTL  and  show  its  utility  in  formulating  algorithms 
for  falsification  of  HyperSTL  specifications.  We  demonstrate  how 
we  can  specify  and  falsify  HyperSTL  properties  for  two  case  studies 
involving  automotive  control  systems. 

ACM  Reference  format: 

Luan  Viet  Nguyen,  James  Kapinski,  Xiaoqing  Jin,  Jyotirmoy  V.  Deshmukh, 
and  Taylor  T.  Johnson.  2017.  Hyperproperties  of  Real-Valued  Signals.  In 
Proceedings  of  MEMOCODE  ’17,  Vienna,  Austria,  September  29-October  2, 
2017,  10  pages. 

DOI:  10.1145/3127041.3127058 

1  INTRODUCTION 

Hyperproperties  were  first  proposed  by  Clarkson  and  Schneider 
to  characterize  properties  of  security  policies  that  cannot  be  de¬ 
fined  over  individual  traces,  such  as  service  level  agreements  and 
information-flow  properties  [15].  In  this  work,  we  extend  the  no¬ 
tion  of  hyperproperties  to  cover  a  broad  range  of  requirements  for 
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cyber-physical  systems  (CPSs),  and  we  present  a  taxonomy  of  hy¬ 
perproperties  used  to  address  security  and  control  design  concerns 
for  CPSs.  Also,  we  provide  practical  techniques  for  automating  the 
process  of  testing  hyperproperties  for  CPSs. 

In  contrast  to  trace  properties  expressed  over  individual  execu¬ 
tion  traces,  hyperproperties  are  defined  over  multiple  execution 
traces.  For  example,  one  execution  of  a  system  cannot  be  checked 
against  a  service  level  agreement  property  such  as  “the  average 
time  elapsed  between  a  user’s  request  and  response  over  all  executions 
should  be  less  than  1  second”;  the  property  can  only  be  evaluated 
over  all  system  execution  traces.  Moreover,  we  can  consider  an 
information-flow  policy  of  noninterference  specified  as  “for  all  pairs 
of  traces  of  a  system  that  have  the  same  low-level  security  inputs, 
they  will  also  have  the  same  low-level  security  output”  [22,  41].  This 
noninterference  property  is  a  hyperproperty  as  it  is  expressed  over 
all  pairs  of  traces  of  a  system. 

Hyperproperties  generalize  more  traditional  formal  properties  by 
specifying  relationships  between  disparate  execution  traces,  instead 
of  behaviors  of  individual  execution  traces.  Traditional  logics  that 
consider  traces  individually,  such  as  LTL,  cannot  be  used  to  specify 
hyperproperties,  and  thus,  hyperproperties  are  more  expressive. 
Logics  such  as  CTL  and  CTL*  allow  properties  over  multiple  paths 
of  a  computation  tree,  but  they  do  not  permit  comparisons  between 
the  paths  themselves.  Instead,  to  express  and  efficiently  check 
hyperproperties,  Clarkson  et  al.,  introduced  notions  of  HyperLTL 
and  HyperCTL*  [14].  Both  logics  directly  extend  LTL  and  allow  us 
to  reason  about  more  than  one  execution  trace  at  a  time.  The  main 
difference  between  HyperLTL  and  HyperCTL*  is  that  the  former 
requires  trace  quantifiers  appearing  at  the  beginning  of  a  formula, 
but  the  latter  allows  us  to  specify  them  within  a  formula. 

Although  hyperproperties  are  well  studied  in  the  context  of 
security  policies  for  software  systems,  hyperproperties  have  not 
been  explored  for  CPSs.  For  a  CPS  that  includes  stochastic  factors 
such  as  noise,  environment  disturbance,  or  transducer  inaccuracies, 
it  is  realistic  for  design  engineers  to  expect  that  the  system  has 
some  acceptable  performance  in  a  probabilistic  sense  rather  than 
requiring  an  absolute  performance  limit  be  met  for  all  individual 
behaviors.  Acceptable  performances  defined  over  the  averages  of 
settling  time,  overshoot,  undershoot,  or  error  bounds  cannot  be 
specified  and  checked  using  individual  execution  traces;  they  must 
be  quantified  over  all  execution  traces. 

Recently,  security-aware  function  modeling  of  CPSs  has  emerged 
as  an  important  research  topic  in  computer  science  and  system 
verification.  A  CPS,  which  is  an  integration  between  cyber  and 
physical  subcomponents,  can  be  vulnerable  to  both  cyber-based 
and  physical-based  attacks  [5,  19,  39,  48].  For  instance,  consider  a 
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modern  automobile,  which  is  a  complex  CPS  composed  of  many 
computer  units  such  as  an  Engine  Control  Unit  (ECU),  the  Trans¬ 
mission  Control  Module  (TCM),  and  an  Electronic  Brake  Control 
Module  (EBCM),  all  interacting  with  the  physical  world  via  sensors 
and  actuators.  Cyber-based  attackers  can  gain  access  to  the  com¬ 
munication  channels  of  a  modern  automobile  through  wireless  or 
in-vehicle  networks.  As  a  result,  attackers  can  infiltrate  an  ECU 
or  EBCM  to  stall  the  engine  or  disable  the  brake  system  [30,  45], 
An  alternative  method  of  attack  involves  gaining  physical  access 
to  the  system,  for  example  by  manipulating  the  signals  processed 
by  the  sensors  (known  as  sensor  spoofing),  to  compromise  secure 
information  or  to  alter  system  behaviors  [5,  46],  Instances  of  ac¬ 
tive  physical-based  attacks  include  vehicle  braking  system  attacks, 
where  faulty  data  is  injected  into  the  wheel  speed  sensor  of  a  vehi¬ 
cle  to  disrupt  the  braking  function  [48] ,  and  insulin  delivery  device 
attacks,  where  glucose  level  sensor  data  is  corrupted  to  compromise 
the  function  of  the  insulin  delivery  service  [31].  A  passive  physical- 
based  attack,  also  called  a  side-channel  attack,  is  based  on  physically 
observing  the  system  behavior  and  using  leaked  information  to 
gain  insights  into  the  system  implementation  [26,  28,  42],  Some 
well-known  side  channel  attacks  are  power  analysis  attacks  [27], 
timing  attacks  [29],  electromagnetic  attacks  [43]  and  differential 
fault  analysis  attacks  [10]. 

Designing  a  safety-critical  CPS  that  is  entirely  secure  from  both 
cyber-based  and  physical-based  attacks  is  challenging  or  impossi¬ 
ble.  A  reasonable  approach  is  to  iteratively  improve  a  CPS  control 
design  using  a  falsification  technique.  Falsification  is  an  automated 
best-effort  approach  to  identify  system  behaviors  that  violate  a 
given  formal  specification  [40],  The  design  approach  would  be  to 
first  formally  specify  safety  properties  of  a  CPS  that  protect  the 
system  against  possible  cyber-based  and  physical-based  attacks 
using  formalisms  such  as  temporal  logic  and  to  then  iteratively 
improve  the  design  using  falsification,  which  would  automatically 
identify  vulnerabilities  in  the  design.  Despite  the  attractiveness  of 
falsification  techniques,  attacks  for  CPSs  often  need  to  be  defined 
over  multiple  execution  traces  of  the  system,  which  is  something 
that  cannot  be  expressed  or  falsified  using  existing  temporal  log¬ 
ics  such  as  LTL,  MTL,  and  STL.  Thus  we  propose  an  extension  to 
these  logics  that  would  be  compatible  with  the  appropriate  spec¬ 
ifications.  In  this  work,  we  present  a  study  of  hyperproperties 
including  stability,  security  and  safety,  as  applied  to  CPSs.  We 
introduce  several  instances  of  hyperproperties  capturing  relation¬ 
ships  (e.g  input-output  relationships)  between  multiple  traces  of 
a  CPS.  We  extend  the  syntax  and  semantics  of  STL  [17]  to  specify 
hyperproperties  over  dense-time  real-valued  signals,  which  results 
in  a  new  logic  called  HyperSTL.  Basically,  we  add  quantifiers  at 
the  beginning  of  an  STL  formula  to  express  relationships  between 
multiple  traces.  We  also  introduce  a  testing  algorithm  based  on 
a  fragment  of  HyperSTL  and  apply  it  to  find  falsifying  traces  for 
hyperproperties  of  industrial  Simulink  models.  Moreover,  we  pro¬ 
vide  a  discussion  on  the  feasibility  of  falsifying  or  verifying  various 
classes  of  hyperproperties  for  CPSs. 

Related  work.  The  study  of  hyperproperties  for  CPSs  evaluated 
in  this  paper  was  inspired  by  the  previous  work  of  Clarkson  and 
Schneider,  who  introduced  hyperproperties  to  express  security 
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policies  such  as  secure  information  flows  and  service  level  agree¬ 
ments  [15],  In  [13],  Bryans  et.  al.  presented  a  general  formalization 
of  opacity  policies  that  prevent  observers  from  deducing  the  truth 
value  of  a  predicate;  those  opacity  policies  require  behaviors  to 
be  specified  over  multiple  paths  of  a  system.  In  earlier  work  [37], 
McLean  showed  that  some  “possibilistic”  security  properties  like 
restrictiveness  [35],  noninterference  [22]  and  nondeducibility  [49] 
are  closure  properties  that  cannot  be  expressed  by  individual  exe¬ 
cution  traces.  In  [37],  those  properties  are  specified  with  respect 
to  different  sets  of  trace  contractors  called  selective  interleaving 
functions. 

Following  the  introduction  of  hyperproperties  [15],  Clarkson 
et  al.  introduced  HyperLTL  and  HyperCTL*,  which  are  exten¬ 
sions  to  existing  temporal  logics,  to  express  and  check  classes  of 
information-flow  hyperproperties  [14].  These  logics  extended  LTL 
and  CTL*  by  adding  the  path  quantifiers  that  permit  specifications 
involving  multiple  paths  in  the  system.  Model  checking  algorithms 
and  complexity  of  fragments  of  HyperLTL  and  HyperCTL*  were 
also  given  in  [14],  which  were  then  further  exploited  and  applied 
to  check  some  classes  of  information-flow  hyperproperties  in  [41]. 

Prototype  implementations  of  model  checkers  for  HyperLTL 
and  HyperCTL*,  which  assume  the  system  is  modeled  as  a  Kripke 
structure,  can  verify  some  information-flow  hyperproperties  of  a 
discrete-time  system,  but  extending  that  work  to  check  hyperprop¬ 
erties  defined  over  continuous  traces  is  a  challenging  endeavor. 
For  complex  CPS  models  or  for  models  built  in  frameworks  with 
proprietary  or  otherwise  obfuscated  semantics,  such  as  Simulink® , 
formal  verification  of  hyperproperties  is  effectively  impossible,  as 
no  corresponding  Kripke  structure  may  be  obtained  from  those 
models1.  Alternatively,  an  easier  but  still  difficult  task  is  to  develop 
an  efficient  testing  framework,  which  could  be  used  to  check  hy¬ 
perproperties  for  finite  collections  of  traces  or  could  be  used  to 
falsify  hyperproperties  of  a  CPS  model;  this  is  the  contribution  of 
the  work  presented  herein. 

In  [50],  Xu  et  al.  introduced  a  notion  of  CensusSTL  that  utilizes 
STL  by  adding  an  outer  logic  to  quantify  the  number  of  individual 
agents  of  a  multiagent  system  whose  behaviors  satisfy  an  inner  STL 
formula.  CensusSTL  is  similar  to  the  HyperSTL  proposed  in  this 
paper;  however,  the  former  is  only  able  to  specify  group  behaviors 
from  different  components  of  an  individual  trace  while  the  latter 
allows  us  to  express  relationships  between  multiple  traces. 

The  remainder  of  the  paper  is  organized  as  follows.  Section  2  re¬ 
views  relevant  background.  Section  3  introduces  several  examples  of 
hyperproperties  of  CPSs  including  stability,  security  and  safety.  Sec¬ 
tion  4  presents  the  syntax  and  semantics  of  HyperSTL.  Section  5 
and  Section  6  describe  the  testing  algorithm  for  two  fragments  of 
HyperSTL.  Section  7  applies  the  proposed  approach  to  find  falsify¬ 
ing  traces  for  some  hyperproperties  of  industrial  Simulink  models, 
and  Section  8  concludes  the  paper. 


1Some  have  created  their  own  translation  of  Simulink  models  to  modeling  languages 
with  well-defined  formal  semantics  (for  example,  see  [3,  52]),  but  these  translations 
necessarily  only  handle  a  subset  of  the  Simulink/Stateflow  modeling  language.  This 
is  due  to  the  fact  that  some  Simulink  constructs  correspond  to  behaviors  that  cannot 
be  modeled  using  standard  frameworks  for  hybrid  systems.  One  such  construct  is 
the  Variable  Transport  Delay  block,  which,  roughly  speaking,  corresponds  to  a  delay 
differential  equation,  a  construct  that  is  not  handled  by  standard  modeling  frameworks 
for  hybrid  systems. 
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2  PRELIMINARIES 

In  this  section,  we  review  the  concepts  of  signal,  system,  trace 
property,  falsification,  and  verification. 

Signal.  We  define  a  signal  w  as  a  function  w  :  T  — >  D,  where 
T  C  R>o  is  the  time  domain.  If  D  =  B,  w  is  a  Boolean  signal  whose 
value  is  either  true  or  false,  and  if  D  =  R,  then  we  say  that  the  signal 
is  real-valued.  A  trace ,  w  :  T  — >  B>i  X  .  . .  X  D„  ,  is  a  collection  of  n 
signals,  where  Vf  6  T,w(f)  =  (wj(t),  W2(f), ...,  wn(t)).  Intuitively, 
we  can  consider  w  as  one  execution  trace  of  a  continuous-time 
system  with  n  variables  that  describes  an  evolution  of  the  system. 
In  what  follows,  we  reserve  the  use  of  bold  letters  like  w,  w'  for 
traces  (i.e.,  tuples  of  signals),  while  we  use  lowercase  italicized 
letters  such  as  w;  to  represent  signals. 

System.  We  define  a  deterministic  or  nonstochastic2  cyber-physical 
system  £  as  a  function  mapping  a  given  input  trace  in  (T  — >  Bm) 
to  an  output  trace  in  (T  — »  B").  We  denote  by  [£]  the  set  of  traces 
w  such  that  the  first  m  components  of  w  correspond  to  the  m  input 
signals  for  [£],  and  the  next  n  components  correspond  to  the  n 
output  signals. 

Trace  properties.  A  trace  property  tp  is  a  finite  or  infinite  set  of 
individual  traces.  A  trace  property  is  either  satisfied  or  violated  by 
any  given  set  of  traces  [6,  41].  A  set  of  traces  W  satisfies  the  trace 
property  <p  if  W  C  tp.  As  noted  above,  an  individual  trace  can  have 
several  components,  for  example,  a  trace  could  contain  m  input 
signals  and  n  output  signals  of  a  given  system  £.  We  say  that  the 
trace  property  tp  holds  for  a  system  £  (denoted  as  £  |=  <p)  if  the  set 
of  input-output  traces  compatible  with  the  system  description  is 
contained  in  the  trace  property,  i.e.,  [£]  C  <p. 

Falsification.  Given  a  trace  property  tp  and  a  CPS  £,  the  falsi¬ 
fication  problem  is  to  find  a  non-empty  set  W  C  [£]  such  that 
W  (£  <p. 

Verification.  Given  a  trace  property  <j>,  the  verification  problem  of 
a  CPS  £  with  respect  to  f  is  to  show  that  [£]  C  tj> . 

3  HYPERPROPERTIES  OF  REAL-VALUED 
SIGNALS 

Hyperproperties  generalize  formal  properties  of  a  system  by  con¬ 
sidering  sets  of  sets  of  execution  traces,  instead  of  only  sets  of 
execution  traces. 

Definition  3.1  (Hyperproperty).  Let  S  denote  the  set  of  all  traces.  Let 
the  power  set  of  S  be  written  as  P  =  P(S).  A  hyperproperty  is  any 
subset  of  PiS). 

We  say  a  set  of  traces  W  satisfies  a  hyperproperty  tf>  C  P  if  W  6  <j>. 
Given  a  hyperproperty  (j>  and  a  system  £,  the  falsification  task  is  to 
find  a  non-empty  set  W  C  [£]  such  that  W  g  <f>.  Similarly,  given  a 
hyperproperty  <j)  and  a  system  £,  the  verification  task  is  to  show 
that  [£]  6  <j>. 


2Note  the  contrast  with  stochastic  systems.  In  stochastic  systems,  one  or  more  parts 
of  the  system  have  randomness  associated  with  them;  for  instance,  the  value  of  a 
particular  system  parameter  may  be  drawn  from  a  probability  distribution.  The  key 
difference  is  that  the  stochastic  system  may  not  produce  the  same  output  for  a  given 
input.  Unless  otherwise  specified,  all  the  systems  that  we  consider  in  this  paper  are 
deterministic. 
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In  this  section,  we  introduce  hyperproperties  for  determinis¬ 
tic  systems  to  characterize  properties  such  as  security,  safety,  and 
stability.  We  focus  on  a  class  of  hyperproperties  capturing  relation¬ 
ships  ( e.g .,  the  input-output  relationship)  between  multiple  traces  of 
a  system,  and  we  show  several  examples  of  hyperproperties  related 
to  stability  and  security  for  CPSs.  In  rest  of  this  section,  we  use 
dsupi w,  w' )  to  denote  the  sup-norm  distance  between  traces  w  and 
w',  where  dsup( w,  w')  =  sup(gRa)  ||w(f)  -  w'(f)||. 

•  Robust  behavior  is  a  requirement  that  guarantees  that  small  dif¬ 
ferences  in  system  inputs  result  in  small  differences  in  system 
outputs.  Consider  the  following  property:  “For  all  pairs  of  traces 
of  a  system  with  an  input  difference  less  than  ei,  the  output  differ¬ 
ence  should  be  bounded  by  ef .  Such  a  property  is  a  hyperproperty 
as  it  requires  at  least  two  execution  traces  to  check.  This  hyper¬ 
property  can  be  formally  written  as: 

4>i  =  {W  e  P  |  Vw.w'  e  W  :  dSUp(win,w'in)  <  e\ 

— '  dsup(wout,'wout)  <  e 2 }.  (1) 

This  type  of  property  is  related  to  certain  stability  notions,  such  as 
bounded  input,  bounded  output  (BIBO)  stability  and  the  £2  gain, 
as  these  notions  also  bound  the  variation  in  the  output,  based 
on  bounded  variation  in  the  input.  We  note,  however,  that  the 
robust  behavior  hyperproperty  differs  from  BIBO  stability  and 
the  £ 2  gain.  as  the  robust  behavior  hyperproperty  is  specified 
over  all  pairs  of  execution  traces  while  the  BIBO  and  £2  proper¬ 
ties  are  defined  based  on  individual  traces.  The  robust  behavior 
hyperproperty  is  also  related  to  bisimulation  relations  [18]  and 
conformance-closeness  [2]  for  a  dynamical  system,  as  all  three  of 
these  properties  are  based  on  some  constraints  on  the  distances 
between  multiple  traces.  In  fact,  we  may  specify  bisimulation 
or  conformance-closeness  functions  in  terms  of  hyperproperties. 
Lastly,  we  note  that  the  robust  behavior  hyperproperty  is  per¬ 
haps  most  closely  related  to  Lipschitz  Robustness  of  systems  [23], 
which  bounds  differences  in  output  behaviors  based  on  bounded 
differences  in  input  behaviors,  though  Lipschitz  Robustness  was 
originally  developed  for  timed  input/output  systems  as  opposed 
to  general  CPS  models. 

•  Side-channel  attacks  are  attacks  against  cryptographic  devices 
based  on  studying  leaking  information  about  the  operations 
they  process,  such  as  power  consumption,  heat  generation,  and 
execution  time.  The  side  channel  attack  is  an  instance  of  an 
inactive  physical-based  attack  that  can  be  used  against  a  CPS  in 
which  some  physical  behaviors  are  observable.  Attackers  can 
deduce  the  working  principle  of  a  system  without  either  access 
to  the  system  itself  or  an  understanding  of  the  internal  operation 
of  the  system.  For  example,  attackers  can  analyze  an  abnormal 
change  in  the  power  consumption  of  an  integrated  circuit  while 
an  encryption  process  is  being  executed  and  then  reconstruct 
the  encryption  key  to  access  secret  data  [27,  28].  The  following 
property  permits  side-channel  attacks: 

(j>2  =  {TV  6  P  I  3w  6  W  :  Vw'  6  W  :  (dsup( w,  w')  >  0 

A  Power(w(t))  >  cj)  =>  Power(w'(t))  <  C2},  (2) 

where  Powerfw(t))  represents  the  power  consumption  corre¬ 
sponding  to  w  over  time,  and  ci ,  C2  are  arbitrary  constants  such 
that  Ci  >  C2-  A  system  that  satisfies  this  property  allows  an 
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attacker  to  detect  that  a  particular  behavior  has  occurred  (w  in 
Formula  2)  by  monitoring  the  power  associated  with  the  behav¬ 
ior.  The  property  is  a  hyperproperty  as  it  is  expressed  in  terms 
of  multiple  traces.  To  ensure  the  safety  of  a  system  from  the 
power-monitoring  attack,  the  system  should  satisfy  -i0 2.  We 
note  that  other  classes  of  side-channel  attacks  such  as  timing 
attacks,  electromagnetic  attacks,  and  differential  fault  analysis 
attacks  can  be  specified  using  properties  similar  to  Formula  2. 

•  Robust  control  invariance  is  a  property  that  can  be  used  to  syn¬ 
thesize  safe  controllers,  or  more  to  the  point,  can  be  utilized  to 
determine  whether  a  safe  controller  exists  for  systems  with  dis¬ 
turbances  [11],  Informally,  the  property  states  that,  for  a  given 
set  of  behaviors  that  is  deemed  safe,  a  control  action  exists,  such 
that  the  system  remains  within  the  safe  set  for  any  allowable 
disturbance  input.  This  can  be  stated  formally  as  follows: 

03  =  {W  e  P  |  3w  6  W  :  Vw’  S  W  :  (w,  w')  |=  0},  (3) 

where  (w,  w')  |=  0  means  that  the  pair  (w,  w')  satisfies  some 
property  0.  In  this  formulation,  wu(t)  is  the  component  of  w 
that  represents  the  controller  action,  wg(t)  is  a  disturbance  input, 
Wy(f)  is  a  system  output,  and  (w,  w')  |=  0  enforces  both  that 
wu  =  w'  and  w y(t)  6  Q,  where  Q  is  the  set  of  safe  behaviors. 
The  robust  control  invariance  property  is  related  to  fault  data 
injection  (FDI)  attacks,  which  are  active  physical-based  attacks 
where  attackers  try  to  input  faulty  data  into  a  system  to  corrupt 
the  behavior  of  the  controller.  For  example,  attackers  can  spoof 
the  sensors  of  DC  microgrids  by  injecting  false  data  such  as 
the  past  outputs  of  the  sensors  at  previous  time  instants.  This 
instance  of  FDI  attack  is  also  well  known  as  a  replay  attack  [8, 
31,  48].  FDI  attacks  have  been  studied  widely  for  CPS,  and  many 
techniques  have  been  proposed  to  efficiently  detect  those  attacks 
in  the  early  stages  [8,  32,  34],  However,  the  optimal  solution  is 
to  design  a  system  that  can  defend  itself  against  FDI  attacks  [38], 
To  guarantee  that  a  system  can  defend  against  a  sensor  attack, 
given  a  specification  0,  it  must  be  possible  to  choose  a  controller 
that  ensures  that  the  output  of  the  system  always  satisfies  0,  i.e. 
03  must  hold. 

3.1  Beyond  Hyperproperties? 

A  hyperproperty  is  more  expressive  than  a  trace  property  as  it  is 
defined  over  a  set  of  sets  of  traces  and  requires  multiple  traces 
to  check.  If  a  system  is  modeled  as  trace  sets,  one  interesting 
question  to  ask  is  whether  there  are  system  properties  inexpressible 
as  hyperproperties.  For  security  policies,  all  properties  of  trace 
sets  can  be  considered  as  hyperproperties,  so  the  answer  may  be 
negative  [6,  15].  For  CPSs,  there  may  exist  some  properties  that  are 
challenging  to  classify. 

Consider  the  following  property  specifying  the  Lyapunov  stabil¬ 
ity  of  a  dynamical  control  system: 

0Ly  =  {Ve  6  [0,  00),  35  6  [0,  e),  Vw  6  W  : 

||w(0)||  <5  =>  (t  >  0  A  ||w(t)||  <  e)}.  (4) 

Intuitively,  this  property  indicates  that  a  system  is  Lyapunov  stable 
if  for  any  e-ball  around  the  origin,  there  exists  a  8-ball  around  the 
origin  (5  <  e)  such  that  if  the  system  starts  within  the  8-ball,  then 
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Figure  1:  Illustration  of  a  Lyapunov  stable  system. 

it  will  never  leave  the  e-ball  [9],  The  illustration  of  a  Lyapunov 
stable  system  is  shown  in  Figure  1. 

Lyapunov  stability  is  specified  over  the  space  of  parameters  and 
execution  traces,  and  involves  two  alternations  between  universal 
and  existential  quantifiers.  As  we  cannot  check  the  Lyapunov 
stability  with  individual  traces,  it  is  not  a  trace  property;  so  is  it 
a  hyperproperty?  Consider  the  parameters  5  and  e  as  constant 
signals,  and  then  rewrite  Lyapunov  stability  as  follows: 

0Ly  ^  6  P  I  Vw  e  W  :  3w/  S  W  :  Vw"  6  W  : 

IK'U,(°)II  <  ^(0)  =>  (f  >  0  A  ||w"f(t)||  <  we(t))},  (5) 

where  a  trace  w  is  composed  of  two  constant  input  signals  wg, 
and  an  output  signal  wOUf.  By  mapping  parameters  into  constant 
signals,  we  can  express  interesting  properties  of  the  system  as 
hyperproperties.  Then  Lyapunov  stability  is  a  hyperproperty  that 
requires  multiple  traces  to  check;  and  it  can  be  formally  specified 
using  the  HyperSTL  introduced  in  the  next  section.  As  to  the 
original  question  of  whether  all  system  properties  of  interest  can 
be  specified  as  hyperproperties,  we  leave  this  open. 

Remark  3.2  Although  we  focus  on  describing  hyperproperties 
defined  over  real-valued  signals,  we  note  that  there  are  other  hyper¬ 
properties  that  can  be  specified  in  the  context  of  CPSs  as  well.  For 
instance,  the  nondeducibility  property  is  an  important  information- 
flow  security  policy  that  prevents  a  low-level  observer  with  suffi¬ 
cient  knowledge  of  a  target  CPS  from  deducing  high-level  (confi¬ 
dential)  information.  The  nondeducibility  property  is  defined  such 
that  for  each  low-level  input  trace,  there  are  more  than  one  possible 
high-level  input  traces  that  produce  the  same  output.  Intuitively, 
an  attacker  should  not  be  able  to  distinguish  between  permissible 
high-level  behaviors  based  on  low-level  behaviors  [20,  36],  On 
the  other  hand,  the  noninterference  property  is  another  important 
information-flow  security  policy  that  requires  that  high-level  secu¬ 
rity  users  should  not  interfere  with  low-level  security  users.  Intu¬ 
itively,  the  outputs  observed  by  the  low-level  security  users  remain 
unchanged  despite  the  actions  of  the  high-level  security  users  [22]. 
Other  variants  of  the  noninterference  property  such  as  noninfer¬ 
ence  [37],  observational  determinism  [51],  declassification  [44],  and 
quantitative  nonterinference  [47]  are  also  hyperproperties  that  need 
to  be  specified  over  multiple  traces.  Though  the  nondeducibility 
and  noninterference  properties  are  relevant  for  CPS,  in  many  cases 
their  impact  on  and  from  real-valued  signals  is  tenuous,  and  so  we 
do  not  treat  them  further  herein. 
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4  HYPERSTL 

In  this  section,  we  introduce  HyperSTL,  a  temporal  logic  that  can 
be  used  to  specify  a  class  of  hyperproperties  of  real-valued  signals. 
The  syntax  and  semantics  of  HyperSTL  are  naturally  extended  from 
those  of  STL  by  adding  existential  and  universal  trace  quantifiers 
into  STL’s  syntax  to  relate  multiple  execution  traces  [17]. 

Syntax.  Let  v  be  a  trace  variable  from  an  infinite  set  of  trace 
variables  *V.  The  syntax  of  HyperSTL  is  then  defined  as  follows: 

(j>  :=  3 \.<j>  |  Vv.tj)  |  <p 

<p  :=  true  |  pv  |  -op  \  ip  A  ip  \  (pVjtp 

Here,  we  add  a  universal  quantifier  V  and  an  existential  quantifier 
3  to  the  syntax  to  indicate  whether  we  want  to  specify  that  a 
formula  holds  over  all  traces  or  over  at  least  one  trace,  respectively. 
For  instance,  Vv3v' .(/)  means  that  for  any  trace  w  assigned  to 
trace  variable  v  ,  there  exists  a  trace  w'  that  can  be  assigned  to 
trace  variable  v'  such  that  f  holds  on  these  two  traces.  We  define 
II  :  *V  — >  S  as  a  trace  assignment  (i.e.,  a  valuation),  which  is  a 
partial  function  mapping  trace  variables  to  traces,  and  S  is  a  set  of 
all  infinite  traces.  Let  Vj  be  the  projection  of  a  trace  variable  v  along 
its  i1*1  component,  the  projection  of  a  trace  assignment  II  (zy )  maps 
Vi  to  the  i4*1  component  of  a  trace  w  (i.e.,  w ;).  Also,  we  abuse  the 
subscript  notation  of  a  trace’s  component  to  write  its  corresponding 
trace  variable’s  component  in  a  HyperSTL  formula,  e.g.,  wout  is 
represented  by  vout.  A  trace  w  can  be  Booleanized  through  atomic 
predicates  of  the  form  pw  =  /(wi(t),  W2 (t), ...,  wn{t))  >  0,  where 
/  is  a  real-valued  function.  Then,  pv  =  f(n(v)(t))  >  0  represents 
a  Booleanized  atomic  predicate  /./w  if  v  is  instanced  by  w.  Also,  I 
is  an  interval  over  R>o  such  as  [a,  b ),  (a,  b),  (a,  b],  [a,  b ],  (a,  +00), 
or  [a,  +oo),  where  a,  b  are  real  numbers  and  0  <  a  <  b.  If  /  is 
not  specified,  we  assume  that  I  =  [0,  oo).  We  also  allow  Boolean 
operators  V  and  =>  with  their  standard  meaning.  Temporal 
operators  used  in  HyperSTL  formulas  include  always  (□),  eventually 
(O),  and  until  (U),  respectively,  where  Ojtp  =  trueVjfp,  and  dj(p  = 
->0 j~xp.  Note  that  we  use  trace  variables  such  as  v,  v'  to  express 
HyperSTL  formula  and  the  corresponding  traces  represented  by 
these  trace  variables  like  w  w'  to  interpret  the  formula.  Consider 
the  HyperSTL  formula  <j>  :=  3v.Vv'.IH[0  jjdlv  —  v'||  <  1).  This 
property  says  that  there  is  always  a  trace  w,  such  that  for  all  times 
in  the  interval  [0, 1],  every  other  trace  w'  is  at  a  bounded  distance 
of  1  from  w. 

Boolean  Semantics.  A  HyperSTL  formula  satisfied  by  a  set  of 
traces  W  at  a  time  t  is  written  as  IT,  t  \=w  The  validity  judgment 
of  a  HyperSTL  formula  at  a  given  time  t  is  specified  according  to 
the  following  recursive  semantics: 


n,  1 1  =w  3v.0 

iff 

exists  w  6  W  :  w  |=  f  and  II(v)  = 

n,  t  |=jy  Vv.0 

iff 

forall  w  6  W  :  w  |=  (J)  and  IT(v)  = 

n,  t  \=w  ptv 

iff 

/(n(v)(0)  >  0 

n,  1 1 =w  ~'<p 

iff 

n,  t  <p 

II,  t  |=w  <pi  A  <p2 

iff 

II,  t  \=w  <pi  and  II,  t  |=w  <p2 

n,  1 1  =w  <pAJi<p2 

iff 

3ti  6  t  +  I  S.t  n,  ti  |=  Viz  <P2 
and  Vt2  £  [t,  t\\  s.t  IT,  t2  |=w  tp\ 
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Using  HyperSTL,  we  can  express  the  hyperproperties  described 
in  Section  3  over  some  time  interval  [t\,  ^2]  as  follows3. 

•  The  robust  behavior  in  Formula  1  can  be  specified  as: 

(j)[  =  Vv.Vv'.  n[tlj2\(dsup(vin, Vin)  <  e\ 

— *  dsup(youuv0ut)  <  e2).  (6) 

•  The  power-monitoring  attack  in  Formula  2  can  be  written  as: 

<t>2  =  3v.Vv'.  □[tl,t2]((<fsup(v,v')  >  0 

A  Power(v)  >  Ci)  =t>  FWer(v')  <  C2).  (7) 

Furthermore,  we  can  rewrite  the  Lyapunov  stability  specified  in 
Formula  5  as  the  following  HyperSTL  formula 

Ky  =  Vv.3v'.Vv".  (v"ut  <  v's  =>  □(o.oo) Wout  <  ve)-  (8) 

According  to  the  possible  alternation  of  quantifiers  in  a  Hyper- 
STL’s  syntax,  we  classify  the  above  HyperSTL  formulae  into  two 
fragments: 

(a)  alternation-free  HyperSTL  formulae  including  one  type  of 
quantifier,  and 

(b)  k-alternation  HyperSTL  formulae  that  have  k  number  of 
alternations  between  existential  and  universal  quantifiers. 

Thus,  the  robust  behavior  property  can  be  expressed  using  alternation- 
free  HyperSTL  while  the  power-monitoring  attack  property  can 
be  specified  using  1 -alternation  HyperSTL.  The  Lyapunov  stabil¬ 
ity  property  is  more  complex  as  it  must  be  expressed  using  2- 
alternation  HyperSTL. 

Falsification  or  Verification  of  Hyperproperties?  We  have  in¬ 
troduced  several  classes  of  hyperproperties  for  CPSs  and  a  temporal 
logic  approach  to  express  them.  Next,  we  investigate  whether  we 
can  falsify  or  verily  those  hyperproperties  using  existing  methods. 
Hyperproperties  are  more  complex  and  expressive  than  traditional 
properties,  and  performing  falsification  and  verification  for  hyper¬ 
properties  is  harder,  in  many  cases.  Despite  this,  we  observe  that 
certain  classes  of  hyperproperties  can  be  falsified  or  verified.  For 
instance,  we  can  falsify  an  alternation-free  HyperSTL  formula  that 
contains  a  universal  quantifier  (e.g.,  the  robust  behavior  hyperprop¬ 
erty),  and  we  can  verify  an  alternation-free  HyperSTL  formula  that 
contains  an  existential  quantifier.  For  the  class  of  hyperproperties 
that  includes  alternating  quantifiers,  falsification  or  verification 
are  often  undecidable  unless  we  impose  some  assumption  about 
the  sets  of  execution  traces  (e.g.,  quantified  over  some  finite  set  of 
traces  with  bounded  time). 

4.1  t-HyperSTL 

We  introduce  t-HyperSTL  as  a  fragment  of  HyperSTL  in  which  a 
nesting  structure  of  temporal  logic  formulas  involving  different 
traces  is  not  allowed.  For  example,  a  formula  Vv.3vr.n|-o  2]V  > 

1  =>  Oji  2j  vr  >  2  is  allowed  but  a  formula  Vv.3v'.IH[q  2](v  > 

1  =>  0[i  2]v'  >  2)  is  not  allowed.  Also,  t-HyperSTL  restricts 
the  until  operator  to  be  specified  over  an  individual  trace,  e.g.,  t- 
HyperSTL  does  not  allow  the  formula  Vv.Sv'^v  >  l)U[o,l](v/  >  2). 

Inherited  from  the  syntax  of  HyperSTL,  t-HyperSTL  formulae 
are  also  classified  into  alternation-free  and  k-alternation  types. 

3  For  a  robust  control  invariance  hyperproperty,  an  instance  of  the  corresponding 

HyperSTL  formula  will  be  shown  in  Section  7.2. 
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t-HyperSTL  suffices  to  express  the  class  of  hyperproperties  formu¬ 
lated  in  Section  3,  and  its  corresponding  semantics,  which  is  more 
restrictive  than  that  of  HyperSTL,  allow  us  to  perform  falsification 
for  these  hyperproperties. 

Quantitative  Semantics.  The  quantitative  semantics  of  t-HyperSTL 
reflects  the  robustness  satisfaction  of  a  t-HyperSTL  formula.  It  is  a 
natural  extension  of  those  for  STL  [17,  33].  Given  J  is  a  real-valued 
function  of  a  formula  <p,  a  trace  assignment  IT,  a  trace  variable  v, 
and  a  time  f,  the  quantitative  semantics  of  t-HyperSTL  is  defined 
inductively  as  follows: 

X(<p ,  IT,  3v,  t )  =  max  x(<P,  n(v)  =  w,  t) 
weW 

X(<p ,  IT,  Vv,  t)  =  min  y(<»,  II(v)  =  w,  t) 
weW 

x(pv  >  o,  n,  v,  t)  =  pv 

x(~“P >  n,  v,  t)  =  ~x(tp,  n,  v,  t ) 

x(<pt  A  <P2,  n,  v,  t)  =  min  (x(ipi ,  n,  v,  t),  x(n ,  n,  v,  0) 

v,  f)  =  sup  min  (^(<p2,n,  v,  ti), 

tl€t+I  inf  *(<pi,n,v,f2)) 
f2e[f,  fi] 

5  FALSIFYING  ALTERNATION-FREE 
T-HYPERSTL 

We  first  consider  the  falsification  of  alternation-free  t-HyperSTL 
formulae.  This  fragment  of  HyperSTL  is  expressive  enough  to 
capture  abroad  range  of  hyperproperties  specifying  input-output 
relationships  over  all  pairs  of  execution  traces.  We  use  a  translation 
scheme  called  self-composition  [7],  which  allows  us  to  falsify  an 
alternation-free  t-HyperSTL  formula  that  includes  only  universal 
quantifiers  using  a  robust  testing  method  for  a  normal  STL  formula. 
Then,  given  an  alternation-free  t-HyperSTL  that  includes  universal 
quantifiers,  we  attempt  to  find  a  set  of  falsifying  traces  for  CPSs 
corresponding  to  this  formula. 

Falsification  algorithm.  The  procedure  that  addresses  the  falsifi¬ 
cation  problem  of  a  system  X  with  respect  to  a  given  hyperproperty 
(ph  over  a  time  duration  T  is  shown  in  Algorithm  1,  and  further 
interpreted  as  follows. 

□  We  first  transform  the  alternation-free  t-HyperSTL  formula  <pi x 
into  the  equivalent  STL  formula  (psTL ■ 

□  We  then  call  a  function  NewSystemGen  to  generate  a  new 
model  that  contains  copies  of  the  original  system.  The  number 
of  copies  is  equal  to  the  number  of  quantifiers  of  the  formula 
<Ph- 

□  Then,  we  apply  existing  falsification  mechanisms  for  an  STL  for¬ 
mula  such  as  Breach1  [16]  to  compute  the  minimum  robustness 
value  Xmin  of  the  system  X'  according  to  tpsTL ■  Breach  allows 
us  to  parametrically  generate  different  input  signals  over  a  pa¬ 
rameter  space.  For  example,  parameters  can  represent  control 
points,  and  an  input  signal  can  be  created  using  interpolation 
between  these  points.  If  Xmin  is  negative  we  return  the  optimal 
set  of  parameters  0y  6  0  that  produces  a  falsifying  behavior. 


4Breach  [16]  is  a  tool  that  applies  a  best-effort  approach  to  automatically  check  whether 
a  system  satisfies  a  given  STL  formula. 
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Algorithm  1  Falsification  of  alternation-free  t-HyperSTL 

1  Require:  a  system  £,  a  parameter  space  0, 
a  t-HyperSTL  formula  <ph ,  a  time  duration  T, 

3  a  maximum  number  of  simulations  N 

begin 

5  (psTL  <—  HyperSTL2STL(<p/,)  //  transform  specification 
57  <—  NewSystemGen(£,  (ph)  //  transform  model 
7  Xmin,  0/  «-  FalsifySTL(X',  ipsTL,  ®,  T,  N) 
if  Xmin  <  o  then 
9  return  By- 

end 

11  end 


We  note  that,  unlike  formal  verification,  performing  falsification 
cannot  ensure  a  system  is  always  safe;  even  if  falsification  fails  to 
identify  a  falsifying  behavior,  a  counter-example  may  still  exist. 

Example  5.1.  Consider  a  mechanical  mass-spring  damper  system 
whose  dynamics  are  defined  by  the  second-order  ordinary  differen¬ 
tial  equation: 

x(t)  +  2  x(t)  +  5  x(t)  =  3  F(t),  (9) 

where  x  is  the  vertical  position  of  the  mass,  and  F  is  the  random 
external  force.  The  robust  behavior  hyperproperty  of  the  system 
is  specified  as  follows:  for  all  pairs  of  traces  of  the  system  with 
the  external  force  difference  less  than  £j  ,  the  output  difference 
should  be  bounded  by  £2;  here  £i  =  0.2  and  £2  =  0.3.  We  apply  the 
Algorithm  1  to  falsify  the  robust  behavior  hyperproperty  for  the 
system  with  a  duration  T  =  10  seconds.  Formula  6  can  be  reduced 
to  the  normal  STL  formula  as  follows: 

(/>M  =  □  [(), 10] (/On  —  el  - '  Pout  ^  £2),  (10) 

where  a  trace  p  =  (pin.  Pout)  of  the  system  X'  captures  the  input- 
output  difference  between  two  traces  w,  w'  of  the  original  system 
X',  e.g.,  pin(t)  =  1 1 W;„(f)  -  wj'n(t)||.  Here,  the  system  X'  con¬ 
tains  two  copies  of  the  mechanical  mass-spring  damper  system  X. 
The  falsification  result  shown  in  Figure  2  illustrates  the  inductive 
checking  procedure  for  the  satisfaction  of  Formula  10  using  Breach, 
where  alw^  10j  is  equivalent  to  □[0,10],  and  the  left  y-axis  denotes 
robustness  degree.  Here,  we  observe  that  the  violation  of  the  robust 
behavior  hyperproperty  of  the  mechanical  mass-spring  damper 
system  occurs  during  the  overshoot  period  of  the  outputs  of  the 
system. 

Remark  5.2  There  is  a  duality  between  addressing  the  falsifica¬ 
tion  problem  of  an  alternation-free  t-HyperSTL  that  only  contains 
universal  quantifiers  and  solving  the  verification  problem  of  an 
alternation-free  t-HyperSTL  that  only  contains  existential  quan¬ 
tifiers.  Given  an  alternation-free  t-HyperSTL  such  as  3w.3x'  ,tpe, 
our  purpose  is  to  extensively  simulate  a  system  and  find  a  single 
pair  of  execution  traces  of  the  system  that  satisfies  (j>e.  Here,  we 
do  not  attempt  to  falsify  the  system,  but  verify  the  system.  Thus, 
this  process  is  dual  to  finding  the  falsifying  traces  of  the  system 
corresponding  to  the  formula  Vv.Vv'  .-a/>e. 

Also,  we  note  that  we  can  leverage  Algorithm  1  such  that  it  includes 
a  parameter  synthesis  approach  to  mine  hyperproperties  for  CPSs, 
as  in  [24,  25],  For  instance,  we  could  use  a  requirement  mining 
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Figure  2:  Falsification  result  of  the  mass-spring  damper  sys¬ 
tem.  The  counterexample  pair  of  traces  found  by  Breach  for 
the  robust  behavior  hyperproperty. 

approach  to  automatically  infer  appropriate  values  for  the  ei  and 
£2  variables  in  Formula  10. 

6  FALSIFYING  K- ALTERNATION  T-HYPERSTL 

Falsifying  k-alternation  t-HyperSTL  formulas  is  a  challenging  task, 
as  it  requires  us  to  examine  all  execution  traces  of  a  system.  Con¬ 
sider  a  1-alternation  t-HyperSTL  formula  such  as  3v.Vv'.0;  falsify¬ 
ing  a  system  for  this  property  is  as  hard  as  verifying  the  system, 
since  we  need  to  show  that  for  all  traces  w  6  S,  there  exists  a  trace 
w'  that  the  formula  (J>  is  violated,  where  S  is  an  infinite  set  of  traces. 
It  is  even  more  difficult  to  perform  falsification  for  CPSs  whose 
dynamics  evolve  continuously  over  time.  Furthermore,  if  a  hyper¬ 
property  contains  more  than  one  alternation  of  quantifiers  (e.g.  the 
Lyapunov  stability  property),  the  falsifying  algorithm  may  suffer 
an  exponential  growth  in  complexity.  Despite  this,  if  we  assume 
a  CPS  can  be  modeled  by  a  finite  set  of  traces,  we  can  develop  a 
falsifying  algorithm  for  the  system  that  can  prove  or  disprove  <j>. 

In  general,  there  may  not  exist  a  unique  answer  to  the  question 
of  whether  we  can  verify  or  falsify  a  system  with  respect  to  the 
formula  Bv.Vv'.^  using  finite  simulations.  We  can  consider  several 
possible  answers  for  that  question  as  follows. 

□  Case  1:  if  both  w,  w'  belong  to  some  infinite  set  of  traces,  then 
we  can  neither  verify  nor  falsify  ijj. 

□  Case  2:  if  w  belongs  to  an  infinite  set  of  traces  and  w'  belongs 
to  a  finite  set  of  traces,  then  we  cannot  falsify  but  we  can  verify 

□  Case  3:  if  w  belongs  to  a  finite  set  of  traces  and  w'  belongs 
to  an  infinite  set  of  traces,  then  we  cannot  verify  but  we  can 
falsify  <j>. 

□  Case  4:  If  both  w  and  w'  belong  to  a  finite  set  of  n  traces,  we 
are  able  to  verify  the  system  with  n  simulations  as  well  as  falsify 
the  system  with  —  simulations. 

We  note  that  in  all  of  the  cases  that  we  are  able  to  falsify  the  system 
corresponding  to  the  formula  3v.Vv'.^>  with  finite  simulations,  we 
can  apply  Algorithm  1  to  transform  the  falsification  problem  to 
another  equivalent  problem  that  uses  a  traditional  STL  specification. 


Table  1:  Feasibility  of  solving  the  falsification  and  verifica¬ 
tion  problems  for  properties  and  hyperproperties  expressed 
using  STL  and  k-alternation  t-HyperSTL  under  two  assump¬ 
tions:  Al)  using  finite  simulation  and  A2)  applying  a  verifi¬ 
cation  oracle  that  can  do  reachability  analysis  with  respect 
to  the  last  quantifier. 


Type 

At:  Finite  Simulation 

A2  :  Verification  Oracle 

Falsification 

Verification 

on  the  Last  Quantifier 

V 

Yes 

No 

- 

3 

No 

Yes 

- 

V3 

No 

No 

V 

3V 

No 

No 

3 

V3V 

No 

No 

V3 

3V3 

No 

No 

3V 

The  falsification  procedure  is  similar  to  solving  the  falsification 
problem  of  alternation-free  t-HyperSTL. 

For  the  case  that  both  execution  traces  of  a  system,  w  and  w', 
belong  to  some  infinite  sets,  and  if  we  have  a  verification  oracle  to 
address  the  last  quantifier  (e.g.,  by  conservatively  estimating  the 
set  of  possible  system  behaviors,  under  certain  conditions),  we  can 
either  falsify  or  verify  the  system.  Given  a  set  of  initial  states,  a 
verification  oracle  can  be  a  method  that  mathematically  overap¬ 
proximates  the  reachable  set  of  the  system  or  a  simulation-based 
technique  [1,  21]  that  may  verify  the  system  with  finite  simulations. 

Alternatively,  for  a  hyperproperty  that  requires  two  or  more 
alternations  of  quantifiers  to  express,  even  if  we  have  a  verification 
oracle  corresponding  to  the  last  quantifier,  we  can  neither  falsify 
nor  verify  a  system.  Using  a  verification  oracle,  the  feasibility  of 
addressing  the  falsification  and  verification  problems  associated 
with  a  ^-alternation  t-HyperSTL  formula  is  equivalent  to  that  of  a 
(fc  -  l)-alternation  t-HyperSTL  formula;  this  is  shown  in  Table  1. 
We  emphasize  that  any  hyperproperties  for  general  CPSs  that  are 
as  complex  as,  or  more  complicated  than  Lyapunov  stability,  are 
not  verifiable  or  falsifiable  without  reasonable  restrictions  on  sets 
of  execution  traces. 

7  CASE  STUDY 

In  this  section,  we  introduce  two  proof-of-concept  case  studies  in 
the  domain  of  automotive  control  systems:  a)  an  industrial-scale 
Simulink  model  of  a  closed-loop  airpath  control  (APC)  system  and 
b)  a  Simulink  model  of  a  fault-tolerant  fuel  (FTF)  control  system.  We 
will  demonstrate  how  to  apply  the  testing  framework  of  HyperSTL 
built  on  top  of  Breach  to  falsify  the  robust  behavior  hyperproperty 
of  the  APC  system,  and  the  robust  control  invariance  hyperproperty 
of  the  FTF  system  under  FDI  attacks. 

7.1  Airpath  Control  Model 

We  use  a  prototype  APC  system  to  evaluate  the  capability  of  our 
proposed  method  on  an  industrial  control  system.  The  APC  is  a  key 
subsystem  for  a  hydrogen  Fuel-Cell  (FC)  vehicle  powertrain.  The 
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Figure  3:  Falsification  result  of  the  APC  system.  The  coun¬ 
terexample  pair  of  traces  found  by  Breach  for  the  robust  be¬ 
havior  hyperpropperty. 

purpose  of  the  APC  is  to  regulate  the  air  flow  rate  into  the  FC  stack 
using  multiple  actuators.  The  FC  stack  generates  electrical  power 
for  the  vehicle  using  a  mixture  of  air  and  hydrogen.  The  FC  stack 
only  operates  under  restricted  conditions,  such  as  temperature, 
pressure  and  moisture  level  within  the  stack.  An  excess  of  moisture 
in  the  stack  will  impede  the  performance  while  moisture  deficiency 
could  permanently  damage  the  FC  stack.  Thus,  to  achieve  high 
performance  while  still  operating  the  system  in  a  safe  regime,  the 
controller  is  required  to  accurately  regulate  the  air  flow  rate. 

The  closed-loop  Simulink  model  of  the  APC  system  is  complex; 
it  contains  more  than  7,000  Simulink  blocks  such  as  integrators,  sat¬ 
urations,  S-Function  blocks,  lookup  tables,  and  data  store  memory 
blocks.  The  model  has  two  input  signals  including  i)  the  ambient 
temperature  and  ii)  the  fuel  cell  current  request  (FCI).  Details  of  the 
system,  such  as  units  and  expected  signal  ranges,  are  suppressed  due 
to  proprietary  concerns.  Intuitively,  an  FCI  value  is  proportional 
to  the  desired  torque  requested  by  the  driver,  which  is  ultimately 
based  on  the  accelerator  pedal  angle.  The  output  of  the  APC  system 
is  an  air  flow  rate  (AFR).  The  purpose  of  the  controller  model  is  to 
regulate  the  AFR  to  some  desirable  reference  value.  To  ensure  the 
APC  system  works  properly,  for  some  small  perturbations  of  the 
ambient  temperature  and  FCI  values,  the  differences  in  AFR  values 
should  be  bounded  within  a  desirable  range.  In  other  words,  to 
avoid  unexpected  changes  in  the  air  flow  rate  at  the  inlet  of  an  FC 
stack,  which  may  cause  undesirable  behavior,  the  system  should 
satisfy  the  robust  behavior  hyperproperty.  The  robust  behavior 
hyperproperty  of  the  APC  system  can  be  formalized  as  follows, 

(pAPC  =  {W  6  P  |  Vw,w'  S  W  : 

(dsup^Wtemp*  wtemp )  —  A  dsup{\^pci^pQi)  —  £2) 

=>  dsup(^AFR,^AFR)  -  e 3)}’  (H) 

which  can  be  translated  to  the  following  STL  formula  using  Algo¬ 
rithm  1  to  perform  the  falsification  task, 

0 APC  =  n[0,T]((ptemp  <  e1  A  pFCI  <  £2)  =>  PAFR  ^  e3)> 

(12) 


Nguyen  et  al. 

where  a  trace  w  is  composed  of  the  temperature  and  FCI  input 
signals  Wtemp  and  w pci  respectively,  and  the  AFR  output  signal 
m>AFR ■  Here,  we  create  a  new  model  including  two  copies  of  the 

original  APC  system;  and  a  trace  p  =  (ptemp,  PFCI,  PAFR)  of  the 
new  model  captures  the  input-output  difference  between  two  traces 
w,  w'  of  the  original  model,  for  instance,  ptemp(t)  =  W'Wtemp(t)  - 

'WON- 

The  result  of  falsification  of  the  robust  behavior  hyperpropety  of 
the  APC  system  is  shown  in  Figure  3,  where  the  blue  lines  present 
the  distance  signals  ptemp ,  PFCI<  PAFR  respectively,  and  the  red 
lines  demonstrate  their  corresponding  bounds.  Here,  the  parameter 
values  selected  by  a  design  engineer  are  normalized  to  0.5.  That 
is,  €\  =  0.5,  £2  =  0.5,  and  £3  =  0.5.  The  sampling  time  is  0.001024 
seconds  and  the  simulation  time  T  is  10  seconds.  For  proprietary 
reasons,  we  normalize  the  quantities  and  suppress  the  units  for 
the  data  shown  in  the  figure.  The  counterexample  pairs  of  traces 
reported  by  Breach  demonstrate  a  behavior  where  the  output  dif¬ 
ference  exceeds  its  allowed  bounds  when  the  input  differences  are 
still  less  than  their  given  thresholds,  which  is  a  violation  of  For¬ 
mula  12.  Finding  this  counter-example  is  significant,  as  it  can  help 
automotive  control  engineers  to  improve  the  controller  design  to 
eliminate  such  an  undesirable  behavior  of  the  APC  system. 

7.2  Fault-tolerant  Fuel  Model 

We  consider  a  fault-tolerant  fuel  (FTF)  model  that  includes  both 
Simulink  blocks  and  Stateflow  charts5.  The  model  has  two  external 
input  signals,  engine  speed  and  throttle  command,  and  one  output 
signal,  which  is  the  effective  air-fuel  ratio  inside  the  combustion 
chamber.  The  model  also  contains  four  sensors  measuring  throttle 
angle,  engine  speed,  the  amount  of  residual  oxygen  in  the  exhaust 
gas  (EGO),  and  the  manifold  absolute  pressure  (MAP).  The  controller 
has  three  different  control  strategies:  a  normal  operation  mode, 
which  is  used  when  no  sensor  faults  are  present,  a  fault  mitigation 
mode,  which  is  used  when  one  sensor  fault  has  occurred,  and  a 
mode  that  disables  fuel  control,  which  is  used  when  two  or  more 
sensor  faults  are  detected.  We  only  consider  the  normal  and  fault 
mitigation  modes  for  this  example.  The  goal  of  the  controller  is  to 
regulate  the  air-fuel  ratio  output,  denoted  as  A,  so  that  it  remains 
within  a  desirable  range,  despite  a  failure  in  at  most  one  sensor. 

In  this  case  study,  we  evaluate  the  ability  of  the  FTF  controller 
to  tolerate  an  engine  speed  sensor  fault.  In  the  original  version  of 
the  model,  a  speed  sensor  fault  consists  of  the  speed  sensor  output 
being  set  to  0.0  rad/sec;  the  controller  detects  the  fault  when  the 
sensor  reading  equals  0.0.  In  the  modified  version  that  we  use,  we 
do  not  fix  the  controller  mode  based  on  the  sensor  reading,  but 
instead  we  evaluate  the  controller  performance  when  either  the 
normal  or  fault  mitigation  modes  are  selected.  In  the  modified 
version  of  the  model  that  we  use,  a  speed  sensor  fault  consists  of  a 
sensor  output  producing  a  fixed  but  randomly  selected  value  in  the 
sensor  range  [0,  620]  rad/sec.  This  kind  of  sensor  fault  could  occur 
when  an  attacker  uses  a  sensor  spoofing  approach  to  inject  incorrect 
measurements  into  the  sensor  readings  or  when  a  real  fault  occurs 
in  the  speed  sensor.  We  use  the  robust  control  invariance  property 
to  specify  desired  controller  performance  in  the  presence  of  the 

5We  use  a  modified  version  of  the  FTF  model  available  at  https ://www.mathworks. 
com/help/simulink/ examples/modeling-  a-  fault- tolerant-  fuel-  control-  system.html 
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indicated  class  of  sensor  faults: 

(j)FTF  =  Bv.Vv'.n  [T9oo](dsup(vu,v'u)  =  0 

=>  0.8Arey  <  <  1.2 Arey),  (13) 

where  Arey  is  the  reference  value  of  the  air-fuel  ratio  A,  and  r  is  the 
settling  time.  Here,  a  trace  variable  v  can  be  mapped  to  a  trace  w 
composed  of  the  controller  input  wu  corresponding  to  a  controller 
mode  decision,  a  disturbance  representing  the  fixed  random 
sensor  input  injected  into  the  speed  sensor,  and  an  output  wA.In 
general,  we  cannot  falsify  Formula  13  according  to  the  discussion 
shown  in  Table  1;  however,  for  systems  like  the  FTF  model  that 
have  a  finite  set  of  control  strategies,  we  can  effectively  perform 
falsification  by  creating  a  new  model  that  contains  copies  of  the 
original  system,  one  copy  for  each  control  mode  (two  copies,  in  this 
case).  The  external  input  (the  speed  sensor  reading)  is  connected  to 
each  of  the  copies  of  the  model.  The  specification  </>ftf  is  converted 
to  the  following  equivalent  formula  in  standard  STL: 

<j>FTF  =  Vwd.n[r  oo](0.8Arej  <  wx1  <  1-2 Xref 

V  0.8Arey  <  1VA2  <  1.2Arey),  (14) 

where  wA]  and  wA,  are  the  air-fuel  ratios  of  the  first  and  second 
copies  of  the  model.  We  note  that  Formula  14  is  arrived  at  by  apply¬ 
ing  the  quantitative  semantics  provided  in  Sec.  4;  the  disjunction 
in  Formula  14  appears  due  to  the  3  quantifier  in  Formula  13,  which 
effectively  applies  a  max  operator  over  the  two  available  control 
modes.  The  formula  </>ftf  can  be  tested  using  the  falsification 
methods  for  traditional  STL  available  in  Breach. 

Figure  4  illustrates  the  falsification  result  of  the  FTF  model.  The 
blue  lines  correspond  to  a  simulation  trace  representing  the  falsi¬ 
fying  behavior,  the  green  line  illustrates  an  instance  of  the  correct 
speed,  and  the  red  lines  represent  the  error  bound  of  A,  where 
r  =  10  seconds,  T  =  50  seconds,  and  A rej-  =  14.6.  Based  on  the 
results,  we  can  conclude  that  there  exists  a  trace,  which  includes 
outputs  wAl  and  wA2  that  both  evolve  beyond  the  tolerance  bound 
regardless  of  whether  the  controller  operates  in  the  normal  mode 
or  the  fault  mitigation  mode  (i.e.,  the  performance  requirement 
is  violated  despite  which  control  mode  is  used).  This  experiment 
demonstrates  the  capability  of  using  a  falsification  approach  to 
automatically  test  hyperproperties  for  CPSs. 

8  CONCLUSION  AND  FUTURE  WORK 

In  this  paper,  we  represented  the  first  study  of  the  hyperproperties 
of  CPSs.  We  defined  a  new  temporal  logic,  called  HyperSTL,  to 
express  several  hyperproperties  including  stability,  security,  and 
safety  for  CPSs.  HyperSTL  allows  us  to  effectively  specify  more 
general  requirements  of  CPS  rather  than  STL  as  it  can  express  the 
relationships  between  multiple  execution  traces.  The  testing  frame¬ 
work  of  t-HyperSTL,  a  fragment  of  HyperSTL,  was  also  given  and 
applied  to  falsify  the  robust  behavior  hyperproperty  of  a  hydrogen 
fuel-cell  powertrain  model,  and  the  robust  control  invariance  hy¬ 
perproperty  of  the  fuel  control  model  under  a  fault  data  injection 
attack.  We  also  discuss  the  feasibility  of  performing  the  falsification 
and  verification  for  various  classes  of  hyperproperties  for  CPSs. 
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Figure  4:  A  pair  of  falsifying  traces  found  by  Breach  illustrat¬ 
ing  the  FTF  model  cannot  tolerate  the  fault  under  a  speed 
sensor  fault. 

Future  Work.  We  first  plan  to  introduce  a  library  of  HyperSTL 
fomulae  that  encapsulates  different  general  classes  of  hyperprop¬ 
erties  of  CPS  including  those  presented  in  this  paper.  Second,  the 
falsification  algorithm  of  HyperSTL  proposed  in  the  paper  is  in¬ 
complete  as  it  relies  on  self-composition  (i.e.  making  copies  of  a 
system)  and  only  falsifies  a  restricted  class  of  hyperproperties.  Thus, 
extending  the  falsification  algorithm  to  bypass  self-composition 
to  falsify  more  interesting  hyperproperties  is  planned.  Also,  the 
monitoring  algorithms  of  HyperLTL  recently  proposed  in  [4,  12] 
could  be  applied  to  HyperSTL. 
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Embedded  systems  use  increasingly  complex  software  and  are  evolving  into  cyber-physical  systems  (CPS) 
with  sophisticated  interaction  and  coupling  between  physical  and  computational  processes.  Many  CPS  operate 
in  safety-critical  environments  and  have  stringent  certification,  reliability,  and  correctness  requirements. 
These  systems  undergo  changes  throughout  their  lifetimes,  where  either  the  software  or  physical  hardware  is 
updated  in  subsequent  design  iterations.  One  source  of  failure  in  safety-critical  CPS  is  when  there  are  unstated 
assumptions  in  either  the  physical  or  cyber  parts  of  the  system,  and  new  components  do  not  match  those 
assumptions.  In  this  work,  we  present  an  automated  method  towards  identifying  unstated  assumptions  in  CPS. 
Dynamic  specifications  in  the  form  of  candidate  invariants  of  both  the  software  and  physical  components 
are  identified  using  dynamic  analysis  (executing  and/or  simulating  the  system  implementation  or  model 
thereof).  A  prototype  tool  called  Hynger  (for  HYbrid  iNvariant  GEneratoR)  was  developed  that  instruments 
Simulink/Stateflow  (SLSF)  model  diagrams  to  generate  traces  in  the  input  format  compatible  with  the  Daikon 
invariant  inference  tool,  which  has  been  extensively  applied  to  software  systems.  Hynger,  in  conjunction 
with  Daikon,  is  able  to  detect  candidate  invariants  of  several  CPS  case  studies.  We  use  the  running  example 
of  a  DC-to-DC  power  converter,  and  demonstrate  that  Hynger  can  detect  a  specification  mismatch  where  a 
tolerance  assumed  by  the  software  is  violated  due  to  a  plant  change.  Another  case  study  of  an  automotive 
control  system  is  also  introduced  to  illustrate  the  power  of  Hynger  and  Daikon  in  automatically  identifying 
cyber-physical  specification  mismatches. 

CCS  Concepts:  -Theory  of  computation  — ^Program  specifications;  -Software  and  its  engineering 

— » Dynamic  analysis; 
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Cyber-Physical  Specification  Mismatches 

1  INTRODUCTION 

Systems  interacting  with  their  physical  environments  are  becoming  increasingly  dependent  upon 
computers  and  software,  such  as  in  emerging  cyber-physical  systems  (CPS).  For  instance,  typical 
modern  cars  utilize  hundreds  of  microprocessors,  many  communications  buses,  and  a  complex  inter¬ 
connection  between  sensors,  actuators,  and  processors.  In  the  design  and  development  process  for 
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Fig.  1.  High-level  diagram  of  a  closed-loop  control  system. 

most  engineered  systems,  the  vast  majority  of  resources  are  devoted  to  ensuring  systems  meet  their 
specifications  [7],  However,  in  spite  of  significant  technical  advances  for  designing  verification  and 
validation  such  as  model  checking,  Software/Hardware-In-The-Loop  (SIL/HIL)  testing,  automatic 
test  case  generation  for  software,  and  sophisticated  simulators,  there  still  remain  products  recalled 
across  industries  for  safety  concerns  due  to  software  problems  and  system  integration  between 
the  cyber  and  physical  subcomponents.  The  verification  community  typically  focuses  on  the 
developmental  verification,  where  a  model  of  a  system  is  developed  and  properties  (specifications) 
are  (manually,  semi-automatically,  or  automatically)  checked  for  that  system.  However,  many 
product  recalls  and  safety  disasters  induced  by  software  bugs  are  not  a  result  of  design  errors,  but 
are  the  result  of  either:  (a)  implementation  errors,  or  ( b )  reuse,  upgrade,  and  maintenance  errors. 
Initiatives  like  a  priori  Model-Based  Design  (MBD)  are  important  research  efforts  and  may  someday 
lead  to  synthesizing  provably  correct  implementations  from  specifications.  However,  most  systems 
being  designed  today  still  utilize  a  development  process  that  has  engineers  writing  the  software, 
and  systems  are  integrated  with  numerous  components  in  a  potentially  error-prone  process.  For 
instance,  a  typical  CPS  that  has  been  used  widely  in  control  systems  is  a  closed-loop  feedback 
controller  shown  in  Figure  1,  where  a  plant  describes  physical  changes  of  the  environment  and  a 
controller  represents  cyber/software  computations  corresponding  to  these  changes.  The  physical 
evolution  of  the  plant  can  be  sensed  and  sampled  by  a  sensor,  and  then  fed  into  the  controller.  Based 
on  the  measurement  of  the  plant  provided  by  the  sensor,  the  controller  provides  a  corresponding 
control  signal  to  regulate  the  physical  changes  in  the  plant.  This  control  signal  is  converted  by  an 
actuator  before  sending  it  to  the  plant.  Such  a  system  may  contain  different  possibilities  of  failure 
due  to  the  following  main  reasons:  (a)  the  controller  may  make  wrong  assumptions  about  the  plant, 
sensor  or  actuator.  For  example,  changing  parameters  of  the  plant,  sensor,  or  actuator  without 
updating  the  controller  may  produce  potential  specification  mismatches.  This  controller-reuse 
issue  can  lead  to  safety  failures  such  as  the  Honda  vehicles  recalls  or  the  Ariane  5  flight  501  disaster 
described  in  Section  2.  (b)  The  plant  may  be  influenced  by  uncontrolled  factors  (disturbances) 
from  the  environment,  (c)  the  controller  is  initially  encoded  based  on  wrong  information  about  the 
physical  plant,  ( d )  the  sensor  and  actuator  may  have  conversion  errors,  and  (e)  the  control  conflicts 
may  arise  when  using  a  shared  sensor  and  actuator  network. 

In  this  paper,  we  develop  a  method  to  address  such  challenges  that  arise  in  the  product  evolution 
and  upgrade  process  in  CPS.  Our  proposed  method  enables  dynamic  analysis  using  well-established 
software  engineering  tools  for  large  classes  of  Simulink/Stateflow  (SLSF)  models  that  are  frequently 
used  in  CPS  engineering.  In  particular,  the  method  infers  candidate  invariants  of  SLSF  models. 
Invariants  are  properties  of  a  system  that  should  always  hold,  while  conditional  invariants  may 
hold  at  certain  program  points,  for  example,  at  the  beginning  or  end  of  a  function  call  (pre/post 
conditions).  This  is  important  because  such  models  are  amenable  to  formal  verification  using 
existing  research  tools  and  hybrid  system  model  checkers.  Finding  invariants  can  aid  this  process 
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Fig.  2.  Preliminary  overview  of  the  proposed  methodology  using  Hynger  and  Daikon  to  infer  candidate 
invariants  and  detect  specification  mismatches. 

as  they  represent  potential  abstractions  with  a  possibly  less  complex  representation  than  the  set  of 
reachable  states.  The  SLSF  block  diagrams  may  be  black  box  components,  white  box  components, 
or  even  system  implementations  (such  as  when  SLSF  is  used  in  SIL/FFIL  simulation).  In  the  case 
when  the  underlying  SLSF  models  are  known,  they  may  be  formalized  using  hybrid  automata  [31], 
Candidate  invariants  inferred  with  our  Hynger  (for  HYbrid  iNvariant  GEneratoR)  software  tool 
in  conjunction  with  Daikon  [17,  18]  may  be  formally  checked  as  actual  invariants  using  a  hybrid 
system  model  checker  [20] .  Figure  2  shows  a  preliminary  overview  of  our  proposed  methodology. 
As  a  prelude,  we  just  intuitively  demonstrate  how  Hynger  and  Daikon  can  be  used  to  detect 
specification  mismatches.  The  proposed  framework  will  be  fully  presented  in  Section  5. 

Contributions.  The  primary  contributions  of  this  paper  are:  (a)  the  formalization  of  the  cy¬ 
ber-physical  specification  mismatch  problem,  (b)  a  methodology  for  performing  template-based 
automated  invariant  inference  of  white  box  (known)  and  black  box  (unknown)  CPS  models  using 
dynamic  analysis,  (c)  the  Hynger  software  tool,  which  supports  instrumenting  large  classes  of 
SLSF  diagrams  for  dynamic  analysis  using  tools  like  Daikon,  (d)  a  methodology  for  checking  if 
the  inferred  invariants  are  actual  invariants  by  using  formal  models  of  the  underlying  SLSF  model 
diagrams  and  hybrid  systems  model  checkers  such  as  SpaceEx  [20],  etc.,  (e)  two  proof-of-concept 
CPS  case  studies  using  Hynger  to  automatically  identify  cyber-physical  specification  mismatches. 
These  results  can  be  used  to  help  bridging  the  worlds  of  actual  embedded  systems  software  (e.g., 
detailed  SLSF  diagrams  and  generated  C  code)  with  hybrid  system  models. 

Overall,  this  journal  has  been  substantially  extended  from  our  previous  work  [25].  In  fact,  we 
added  the  formal  definitions  of  cyber-physical  specification  mismatches,  cyber-physical  input- 
output  automata,  and  invariant  checking  problem  to  identify  whether  the  inferred  invariants  are 
actual  invariants.  Moreover,  two  proof-of-concept  CPS  case  studies  including  a  buck  converter  and 
an  abstract  fuel  control  system  are  presented  to  show  the  capability  of  Hynger  tool  in  automatically 
identifying  potential  cyber-physical  specification  mismatches  of  CPSs.  The  experimental  results 
illustrate  the  feasibility  of  using  dynamic  invariant  inference  for  analysis  of  embedded  and  cyber¬ 
physical  systems.  Before  presenting  the  details  of  our  approach,  we  first  illustrate  the  pitfalls  of 
CPS  design  reuse  by  citing  examples  of  critical  mistakes  in  existing,  certified  systems. 

2  CYBER-PHYSICAL  DESIGN  REUSE  AND  UPGRADE 

In  this  section,  we  review  cases  where  CPS  design  reuse  and  upgrade  have  led  to  failures  in  existing 
systems.  This  motivates  the  need  for  our  proposed  method  and  our  Hynger  tool,  which  can  be  used 
to  find  and  formalize  unstated  assumptions  in  CPS. 

A  recent  example  of  a  design-reuse  problem  is  the  National  Highway  Transportation  and  Safety 
Administration  (NHTSA)  recall  of  1.5  million  Honda  vehicles  (including  one  of  the  author’s)  due  to 
Electronic  Control  Module  (ECM)  software  problems  that  could  damage  the  car’s  transmission, 
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resulting  in  possible  stalls.  The  root  cause  of  the  safety  defect  was  the  result  of  a  physical  component 
(a  bearing  in  the  transmission)  being  upgraded  to  an  improved  design  between  different  model- 
year  vehicles  without  appropriate  ECM  software  updates  [38].  This  problem  was  widespread 
because  there  was  a  five  year  delay  before  the  problem  was  identified,  and  it  was  used  across  model 
makes  and  years  (e.g.,  from  2005  -  2010  model  year  Accords,  2007  -  2010  CR-Vs,  and  2005  -  2008 
Elements).  This  difficulty  in  root-cause  analysis  emphasizes  the  point  that  such  problems  are 
probably  underreported,  and  the  reuse  of  components  in  CPS  can  lead  to  widespread  serious 
problems. 

Similar  design-reuse  problems  have  famously  occurred  in  aviation— the  Ariane  5  flight  501  dis¬ 
aster  was  a  result  of  reusing  Ariane  4’s  software  without  appropriate  updates  for  the  increased 
thrust  of  the  new  rocket  [1,  29].  Here,  software  developers  made  an  assumption  about  the  physical 
dynamics  of  the  rocket,  but  the  software  was  reused  from  Ariane  4,  while  Ariane  5  had  greater 
thrust,  so  this  assumption  was  invalid.  Finally,  when  considering  the  future  of  CPS,  the  Defense 
Advanced  Research  Projects  Agency’s  System  of  Systems  Integration  Technology  and  Experimen¬ 
tation  (DARPA  SoSITE)  program  [32]  describes  modularized  military  aviation  systems  which  are 
capable  of  rapid  component  swapping  and  upgrade.  Left  unaddressed,  issues  related  to  unstated 
assumptions  in  components  are  likely  to  get  worse  in  future  CPS,  where  changes  can  occur  in  the 
software  and  hardware. 

Besides  design-reuse  problems,  software  upgrades  without  being  thoroughly  tested  and  validated 
may  result  in  an  epic  system  failure.  One  famous  example  of  this  type  of  problem  is  the  disaster 
of  Mars  Climate  Orbiter  (MCO),  developed  by  NASA’s  Jet  Propulsion  Laboratory  (JPL).  The  root- 
cause  of  this  disaster  was  that  different  parts  of  the  software  developer  team  were  using  different 
units  of  measurements.  In  fact,  one  part  of  the  ground  software  upgraded  by  Lockheed  Martin 
Astronautics  (LMA)  measured  the  thrusters  in  English  units  of  pounds  (force)-seconds  instead  of 
metric  units  of  Newton-seconds  as  defined  in  its  original  Software  Interface  Specification  (SIS) 
used  by  JPL  [28,  51].  Therefore,  the  trajectory  of  the  MCO  was  erroneously  calculated  by  ground 
support  system  computers  using  the  incorrect  thruster  performance  data.  This  type  of  software 
failure  occurred  due  to  the  lack  of  adequate  communication  between  different  parts  of  the  software 
team  and  the  uncovered  issues  of  verification  and  validation  processes  [51]. 


2.1  Related  Work 

The  idea  evaluated  in  this  work,  that  of  inferring  physical  system  specifications  from  embed¬ 
ded  software  in  conjunction  with  physical  system  models  and  evaluating  them  for  mismatches, 
was  inspired  by  previous  work  finding  program  specifications  for  pure  software  systems  [46]. 
Cyber-physical  specification  mismatch  is  closely  related  to  model  inconsistency  [48],  architectural 
mismatch  [21],  and  requirements  consistency  [53].  There  are  many  benefits  of  dynamic  analysis 
such  as  using  implementations  instead  of  models  [17,  18,  46]  to  find  dynamic  program  specifica¬ 
tions  [46],  providing  documentation  over  program  evolution  and  checking  if  specifications  change 
drastically  over  program  evolution,  etc.  For  one,  models  are  not  actually  required  for  analysis,  and 
implementations  may  be  used  [17,  18].  The  benefit  of  executing  a  system  implementation  is  that 
there  are  no  mismatches  between  a  model  (potentially  documentation-based)  and  implementation, 
since  it  is  not  necessary  to  have  a  model  at  all.  The  candidate  specification  generated  may  be 
viewed  as  a  form  of  input-output  abstraction  of  the  actual  implementation.  The  limitation  includes 
results  that  are  unsound  without  additional  reasoning. 

Recently,  Medhat  and  his  collaborators  introduced  a  new  framework  for  inferring  hybrid  au¬ 
tomata  from  black-box  implementations  of  embedded  control  systems  by  mining  their  input/output 
traces  [33].  In  their  work,  the  input/output  training  traces  collected  from  executing  a  system  are 
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clustered  and  then  translated  to  event  sequences.  Under  several  assumptions,  hybrid  automata 
representing  the  behaviors  of  the  system  can  be  inferred  using  the  input/output  correlation.  Al¬ 
though  the  work  suffers  some  limitations,  their  proposed  approach  is  a  proof-of-concept  of  using 
dynamic  analysis  to  infer  the  specifications  of  black-box  systems.  This  work  is  highly  relevant 
to  our  proposed  method  as  there  is  an  analogy  between  inferring  hybrid  automata  and  finding 
a  candidate  invariant  for  a  black-box  system.  In  fact,  both  of  them  can  be  considered  as  doing 
specification  inference  using  dynamic  analysis. 

There  are  also  several  tools  such  as  DepSys  [37]  and  EyePhy  [36]  that  used  both  static  and 
dynamic  analysis  to  detect  and  address  the  control  conflict  due  to  dependencies  when  using 
multiple  CPS  applications.  Particularly,  DepSys  is  a  utility  sensing  and  actuation  infrastructure 
for  a  smart  home  that  can  simultaneously  operate  multiple  CPS  applications.  The  main  novelty  of 
DepSys  is  that  it  provides  a  comprehensive  strategy  to  specify,  detect  and  automatically  address 
the  control  conflicts  between  sensors  and  actuators  used  in  a  home  setting.  Similarly,  EyePhy  is  an 
integrated  system  that  can  detect  dependencies  and  then  perform  a  dependency  comprehensive 
analysis  across  HIL  CPS  medical  applications.  A  built-in  simulator,  HumMod,  in  EyePhy  is  able  to 
model  the  complex  interactions  of  the  human  body  using  more  than  7,800  physiological  variables. 
HumMod  demonstrates  the  model  parameters  and  the  quantitative  relationship  among  them  in 
XML  files  that  makes  it  easier  to  update  the  physiological  models  without  the  recompilation  of  the 
whole  system.  EyePhy  can  be  considered  as  the  first  tool  that  performs  the  dependency  analysis 
across  applications’  control  actions  on  the  human  body.  Additionally,  the  sensor  networks  with 
devices  used  in  smart  homes  or  medical  devices  can  be  built  using  the  family  of  Smart  Transducer 
Interface  Standards  (IEEE  1451).  IEEE  1451  has  been  developed  in  order  to  provide  the  common 
communication  interfaces  for  connecting  transducers  (sensors  or  actuators)  to  their  instrumentation 
systems  or  control  networks  [27],  The  Transducer  Electronic  Data  Sheets  (TEDS)  embedded  in  smart 
transducers  are  memory  devices,  which  store  the  manufacture-related  information  of  the  transducer 
such  as  manufacture  ID,  measurement  ranges,  serial  number,  etc.  Thus,  TEDS  allows  transducers 
to  be  self-identified  and  self-descriptive  to  the  device  networks.  It  also  provides  a  standardized 
mechanism  to  facilitate  the  plug  and  play  of  transducers  with  different  control  networks.  Hence, 
IEEE  1451  enables  the  access  of  transducer  data  through  a  common  set  of  interfaces,  allowing  users 
to  select  transducers  and  networks  for  their  applications.  This  advantage  is  crucial  in  facilitating 
the  device  and  data  interoperability,  detecting  and  resolving  conflicts  due  to  dependencies  when 
concurrently  using  multiple  transducers  in  the  device  networks. 

Finding  specifications  is  a  maturing  field  within  software  engineering  [10,  11,  17,  18,  46],  Daikon, 
which  is  used  by  Hynger,  processes  program  traces  to  generate  invariants  [17,  18].  For  several 
languages  (C,  C++,  etc.),  this  process  is  performed  without  access  to  the  source  code  by  instrument¬ 
ing  the  compiled  program  using  Valgrind  [39].  This  makes  it  difficult  to  use  on  non-x86/x86-64 
platforms  (although  Valgrind  is  gaining  access  to  other  architectures),  which  is  a  serious  limitation, 
as  most  embedded  platforms  utilize  other  architectures  (e.g.,  ARM,  AVR,  PIC,  8051,  MSP430,  etc.). 
Due  in  part  to  these  limitations,  Hynger  instruments  architecture-independent  SLSF  diagrams 
directly.  In  the  long  run,  the  Hynger  tool  is  envisioned  to  take  an  arbitrary  SLSF  model,  instrument 
it,  then  analyze  the  resulting  traces  with  dynamic  analysis  to  identify  broad  classes  of  cyber-physical 
specification  mismatches. 

The  most  closely  related  work  using  Daikon  is  to  find  candidate  invariants  of  hybrid  models  of 
biological  system  [9],  and  this  also  illustrates  a  proof-of-concept  of  using  Daikon  as  a  trace  analyzer 
for  non-purely  software  systems.  Daikon  can  generate  invariants  of  many  forms  for  variables 
and  data  structures,  such  as:  ranges  (a  <  x  <  b),  linear  (y  =  ax  +  b),  variable  ordering  (x  <  y), 
sortedness  of  lists,  etc.  Daikon  works  by  instrumenting  source  code  and/or  compiled  binaries  with 
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changes  that  allow  for  looking  at  variable  values,  then  Daikon  essentially  checks  if  variables  satisfy 
some  template  invariants.  For  instance,  if  an  integer  variable  x  is  observed  to  always  be  smaller 
than  some  number,  say  50,  Daikon  may  generate  a  candidate  invariant  of  x  <  50.  Based  on  many 
advantages  of  using  Daikon  as  a  trace  analyzer  [17,  18],  we  prefer  to  use  Hynger  with  Daikon  to 
infer  candidate  invariants  in  our  proposed  framework.  However,  we  note  that  Hynger  can  generate 
a  trace  file  in  many  input  formats  that  are  compatible  with  other  invariant-inference  tools  using 
dynamic  analysis  not  just  Daikon.  Other  research  tools  like  DySy  [11]  and  commercial  tools  like 
Agitagor  [10]  can  be  used  for  generating  candidate  invariants  for  other  languages. 

3  CYBER-PHYSICAL  SYSTEM  MODELS 

The  approach  presented  in  this  paper  applies  to  the  systems  with  formal  models,  informal  models, 
and  unknown  models/implementations.  The  primary  assumption  is  that  interfaces  to  the  models 
or  systems  are  available  as  SLSF  blocks.  There  are  two  main  categories  of  blocks  appearing  in  an 
SLSF  diagram  that  are  supported  by  our  method,  white  box  and  black  box  systems.  The  white 
box  systems  may  contain:  (a)  known,  informal  models,  ( b )  known,  informal  implementations,  or 
(c)  known,  formal  models  (e.g.,  hybrid  automata,  or  more  precisely,  classes  of  SLSF  diagrams  that 
may  be  converted  to  hybrid  automata  [31]).  The  black  box  systems  may  be  completely  unknown, 
and  may  contain:  (a)  unknown  implementations  (e.g.,  compiled  executable  binaries  with  no  source 
available,  such  as  commercial  off-the-shelf  [COTS]  components  and  other  third-party  systems), 
(b)  unknown  models,  and  (c)  actual  cyber-physical  systems  (e.g.,  embedded  controllers,  networked 
computers,  and  physical  plants,  all  that  may  show  up  in  HIL/SIL  simulations  interfaced  with  SLSF). 

Next,  we  define  a  structure  of  CPS  models  used  in  SLSF.  We  will  not  define  formal  semantics 
of  this  structure  or  SLSF  diagrams  in  this  paper.  However,  in  the  case  where  an  SLSF  diagram  is 
a  white  box  and  formal  semantics  may  be  defined,  a  formal  framework  like  hybrid  input/output 
automata  (HIOA)  [30]  may  be  used  to  specify  the  semantics,  such  as  done  in  the  HyLink  tool  [31]. 
Additionally,  if  an  SLSF  diagram  is  a  white  box  and  linear,  we  may  also  be  able  to  use  SL2SX 
Translator  for  transforming  it  into  a  corresponding  formal  model  [34] .  Interested  readers  can  find 
some  graphical  examples  of  the  translation  in  [31,  34],  Other  formalisms  like  actors  and  hierarchical 
state  machines  are  commonly  used  for  formal  modeling  of  other  diagrammatic  frameworks  similar 
to  SLSF  [2,  8,  52,  54].  Given  a  formal  model  J[  and  candidate  specification  E  (e.g.,  found  using 
Hynger),  we  can  check  if  LA  meets  the  specification,  i.e.,  LA  |=  Z  by  using  a  hybrid  system  model 
checker  like  SpaceEx  [20] .  In  some  instances,  we  know  when  an  SLSF  diagram  corresponds  precisely 
to  a  hybrid  automaton  model  [31],  and  in  these  cases,  we  can  check  if  candidate  invariants  found 
with  Hynger  are  actual  invariants. 

First,  we  define  the  hierarchy  represented  by  SLSF  diagrams. 

Definition  3.1  (SLSF  diagram).  An  SLSF  diagram  is  a  tuple  LA  =  (M,  E,  V),  where: 

•  M  is  a  set  of  blocks  (vertices)  that  represent  block  diagrams  (and  sub-blocks/models), 

•  E  C  M  X  M  is  a  set  of  edges  between  blocks  representing  a  parent-child  hierarchy,  and 

•  V  is  a  set  of  variables,  written  as  V  =  (J  veM  V(u)>  where  V(v)  is  a  set  of  variables  for  each 
block  v  e  M. 

According  to  Definition  3.1,  the  graph  G  =  (M.  E)  defined  by  the  vertices  (blocks)  M  and  edges  E  is 
a  rooted  tree,  where  M  are  block  diagrams  and  E  represents  a  parent-child  hierarchical  relationship 
(e.g.,  sub-modules  and  sub-blocks).  Here,  the  root  (i.e.,  top-level)  block  diagram  of  the  model  is 
the  unique  root  of  the  tree,  which  we  denote  as  root(M).  For  a  block  v  e  M,  the  children  of  v  are 
denoted  as  children(v )  and  defined  as  the  set  of  blocks  {w  €  M  |  w  6  E(v)}.  For  a  block  v  6  M,  the 
parent  of  v  is  denoted  as  parent(v)  and  is  defined  as  the  singleton  set  (w  €  M  \  v  e  children (w)}. 
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Clearly,  parenfiroot(M))  =  0.  For  a  block  v  e  M,  the  ancestors  of  u  are  denoted  as  ancestors(v) 
and  defined  inductively  as  the  set  of  blocks  {¥  e  M  |  »  U  we  children(v)  U  children)  w)}  (or 
equivalently,  as  the  transitive  closure  of  children(v)). 

For  a  block  v  6  M,  the  set  of  variables  of  v  is  V(u)  and  is  partitioned  into  sets  of  input  and  output 
variables,  written  respectively  as  V/(u)  and  Vo(u),  and  we  have  V(u)  =  Vj( v)  U  Vo(f )•  A  variable 
x  e  V(u)  is  a  name  for  referring  to  some  state  of  LA,  and  is  associated  with  a  data  type  denoted 
type(x).  Typical  data  types  are  reals,  floating  points,  arrays,  lists,  etc.  The  valuation  of  a  variable 
x  €  V(u)  is  the  set  of  all  values  it  may  take  and  is  denoted  val(x).  The  state-space  of  LA  is  the  set  of 
valuations  of  all  the  variables  V.  An  element  s  of  the  state-space  is  called  a  state,  and  a  trace  is  a 
sequence  of  states.  The  SLSF  diagram  may  also  have  internal  (local)  variables,  although  they  are 
not  externally  visible,  so  we  do  not  include  them,  as  only  input/ output  interfaces  are  visible  for 
external  observation  and  instrumentation. 

Next,  we  define  CPS  models  that  appear  in  SLSF  diagrams. 

Definition  3.2  (CPS  model).  A  CPS  model  is  an  SLSF  diagram  with  a  set  of  n  typed  variables, 
V  =  {xi,x2, . . .  ,xn},  which  is  classified  into  two  subsets  as  follows. 

•  Vp  =  {aq,  a2, . . . ,  a„p}  is  a  set  of  np  <  n  physical  variables  such  that  their  values  are 
continuously  updated,  and 

•  Vc  =  {/Ji,/?2,  ■  •  ■  ,/3nc}  is  a  set  of  nc  cyber  variables  that  are  discretely  updated,  where 
n-np  +  nc. 

Here,  the  set  of  variables  for  each  block  of  a  CPS  model  is  also  partitioned  into  sets  of  physical  and 
cyber  variables,  V(u)  =  V p(v )  U  Vc(u).  In  practice,  this  may  be  accomplished  with  subtyping  using, 
for  example,  an  overloaded  type  for  floats  or  fixed  points  used  for  approximations  of  real  variables 
(e.g.,  in  C,  typedef  double  physical;  typedef  physical  temperature;).  The 
dynamic  changes  of  the  variables  of  the  CPS  model  may  be  described  using  different  SLSF  blocks 
such  as  S-Function  block,  look-up  table,  etc.  In  case  the  CPS  model  is  a  white-box  and  simple 
enough,  we  may  translate  it  to  a  formal  framework  like  HIOA  (e.g  using  Hylink).  In  fact,  we 
can  specify  a  set  of  real-valued  variables  and  their  dynamic  changes  for  the  converted  formal 
model  based  on  capturing  the  output  variables  from  unit  delay,  integrator,  state-space  blocks  in 
the  corresponding  SLSF  diagram  [3] .  Moreover,  we  note  that  the  input  and  output  variables  are 
disjoint,  and  the  cyber  and  physical  variables  are  disjoint,  although  these  are  not  all  mutually 
disjoint.  Hence,  we  further  classify  the  set  of  variables  V(u)  into  different  types  as  follows. 

Definition  3.3  (Variable  Classification).  For  a  block  v  6  M,  a  variable  x  €  V(u)  is  considered  as: 

•  an  input  cyber  variable  if  x  €  Vc(w)  and  x  €  V/(u), 

•  an  output  cyber  variable  if  x  6  Vc(A)  and  x  €  Vo(w), 

•  an  input  physical  variable  if  x  6  Vp(v)  and  x  €  V/(w),  or 

•  an  output  physical  variable  if  x  e  Vp(v)  and  x  e  Vo(v ). 

We  extend  these  notations  in  Definition  3.3  naturally  to  sets  of  variables  if  all  variables  in  a  set 
of  variables  fall  into  these  classes,  and  will  reference  them  as  such.  An  arbitrary  set  of  variables 
may  not  be  mutually  disjoint  from  each  of  the  input,  output,  cyber,  and  physical  variables.  Thus, 
for  a  set  of  variables  X  C  V,  we  say:  (a)  X  is  cyber-physical  if  there  exist  both  cyber  and  physical 
variables  in  X,  ( b )  X  is  input-output  if  there  exist  both  input  and  output  variables  in  X,  and  (c)  X 
is  cyber  input-output,  physical  input-output,  cyber-physical  input,  or  cyber-physical  output  for  the 
other  natural  permutations. 

Next,  using  these  variable  classes,  we  define  classes  of  SLSF  blocks  appearing  in  SLSF  diagrams. 
For  a  block  v  6  M,  we  say:  (a)  v  is  a  cyber-physical  block  if  there  exist  both  cyber  and  physical 
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variables  in  V(z;),  ( b )  v  is  a  cyber  block  if  there  exist  only  cyber  variables  in  V(u),  and  (c)  v  is  a 
physical  block  if  there  exist  only  physical  variables  in  V(u). 

Cyber-Physical  Variable  Interactions.  Next,  we  will  formalize  a  notion  of  influence  between  cyber 
and  physical  models  and  their  variables.  For  example,  consider  a  typical  closed-loop  plant-controller 
architecture,  where  outputs  of  a  plant  are  sensed,  used  as  inputs  to  a  controller,  and  outputs  of  the 
controller  are  converted  by  actuators  as  inputs  to  the  plant  (and  potentially  disturbances  affect 
everything).  Generally,  we  would  say  the  plant  is  a  physical  model,  the  controller  is  a  cyber  model, 
and  the  sensors  and  actuators  are  cyber-physical  models.  However,  it  is  clear  that  the  physical 
variables  of  the  plant  affect  the  cyber  variables  of  the  controller,  and  vice-versa,  albeit  not  directly, 
but  through  the  transitive  closure  of  input-output  connections  over  all  blocks  in  the  SLSF  diagram. 
We  note  that  this  is  related  to  the  notion  of  tainted  variables  in  program  analysis  that  is  popular 
in  security  [49].  To  formalize  this  notion,  we  specify  interconnections  between  input  and  output 
variables  between  blocks  ueMat  the  same  hierarchical  level  in  the  diagram. 

Input-output  connections  may  only  exist  between  models  with  the  same  parent  (i.e.,  those  in 
the  same  hierarchical  structure).  For  a  block  v  6  M,  we  denote  all  blocks  with  the  same  parent 
as  siblings(v),  which  is  defined  as  the  set  {w  e  M  \  parent(w)  =  parent(v)}.  Output  variables 
of  a  block  v  e  M  may  be  connected  to  input  variables  of  a  block  w  e  M.  Let  Gy  =  (Vv,£v) 
be  a  directed  graph  where  the  vertices  Vy  are  variables  of  blocks  v  €  M  and  the  edges  specify 
the  interconnection  between  output  variables  to  input  variables  for  some  model  w  e  siblings(v), 
and  we  have  £y  C  V(u)  X  V(w).  In  general,  for  a  fixed  block  v  e  M  and  variable  x  e  V(v),  this 
interconnection  relation  is  a  tree,  rooted  at  the  output  variable  x  and  connected  to  possibly  many 
input  variables  of  other  blocks  w  €  M  for  w  +  v.  For  two  blocks  v,w  e  M,  we  say  v  connects  to  w  if 
there  exists  an  output  variable  y  €  Vo(v)  and  an  input  variable  u  €  Vj(w)  with  Ey(u)  =  y,  denoted 
v  w.  For  two  blocks  v,  w  e  M,  we  say  v  has  a  path  tow  if  w  is  in  the  transitive  closure  of  blocks 
that  v  connects  to  (i.e.,  v  c— >*  w),  denoted  v  ^  w.  We  note  that  the  relation  may  have  cycles, 
and  such  cases  arise  in  feedback  control  loops.  For  a  block  v  e  M,  for  an  input  variable  u  e  Vj(v) 
and  output  variable  y  e  Vo(v),  we  say  u  directly  influences  y  if  the  value  of  y  changes  as  a  function 
of  u.1  Finally,  for  two  blocks  v,  w  e  M  such  that  v  ^  w,  for  an  output  variable  y  €  Vo(v)  and  an 
input  variable  u  e  Vj(w),  we  say  y  influences  u  if  there  exists  a  sequence  of  directly  influenced 
variables  between  y  and  u.  Thus,  we  can  see  that  a  cyber  variable  in  one  model  may  influence  a 
physical  variable  in  another  model  (or  even  its  own  model  if  there  is  a  cycle),  and  vice-versa.  The 
software  physical  variables  are  all  cyber  variables  that  are  influenced  by  physical  variables,  and  are 
denoted  Vsp.  Typical  examples  of  software  physical  variables  include  those  used  for  sensed  and 
sampled  measurements,  variables  used  in  feedback  control  calculations,  etc. 

Example  3.4.  Here,  we  describe  a  CPS  case  study  used  throughout  the  remainder  of  the  paper 
for  illustrating  concepts.  The  case  study  is  a  DC-to-DC  power  converter  (like  buck,  boost,  and 
buck-boost  converters)  [40],  all  of  which  have  similar  modeling,  but  we  focus  particularly  on  a  buck 
converter.  The  buck  converter  has  two  real-valued  state  variables  modeling  the  inductor  current  ii 
and  the  capacitor  voltage  Vc-  These  state  variables  are  written  in  vector  form  as:  x  =  [iT;  Vc].  The 
dynamics  of  the  continuous  variables  in  each  mode  m  e  {Open,  Close,  DCM}  are  specified  as  linear 
(affine)  differential  equations:  x  =  Amx  +  Bmu,  where  u  —  V5  is  a  source  voltage.  The  Am  matrices 
consist  of  L  >  0,  R  >  0,  C  >  0  real-valued  constants,  respectively  representing  inductance  (in 
Henries),  resistance  (in  Ohms),  and  capacitance  (in  Farads).  A  buck  converter  takes  an  input  voltage 
of  say  5V  and  “bucks”  or  drops  the  voltage  to  some  lower  DC  voltage,  say  2.5V.  These  circuits 

'internally  the  blocks  may  be  very  sophisticated,  could  represent  complex  physical  systems,  could  be  Turing  complete,  etc., 
so  we  use  this  abstract  notion. 
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are  used  in  many  electronic  devices  (e.g.,  personal  computers,  cellphones,  embedded  systems, 
aircraft,  satellites,  cars).  These  circuits  are  also  used  as  modular  components  in  a  variety  of  novel 
power  electronics  architectures,  such  as  AC/DC  microgrids  and  distributed  DC-to-AC  multilevel 
inverters  [42]. 

The  general  architecture  of  the  buck  converter  that  we  focus  on  consists  of  a  plant  (physical 
system)  model  and  a  controller  (cyber  model/software),  along  with  models  of  actuators  and  sensors 
interfacing  the  plant  and  controller.  A  controller  for  the  buck  converter  may  be  constructed  as  a 
hysteresis  controller,  which  changes  the  mode  of  the  buck  converter  plant  based  on  the  measured 
output  voltage  [22].  In  fact,  the  converter  is  meant  to  transform  a  given  source  voltage  Vs  to 
create  an  output  voltage  Vout  approximately  equal  to  a  desired  reference  voltage  (or  set-point) 
Vref.  To  accomplish  this,  the  switch  controlling  whether  Vs  is  connected  to  the  output  or  not  is 
toggled  depending  on  whether  Vout  >  Vref  or  Vout  <  VTef.  In  practice,  to  avoid  switching  too  often,  a 
hysteresis  band  is  used  and  switches  occur  when  Vout  >  Vref  +  Vtoi  or  Vout  <  Vref  —  Vt 0/.  The  choice 
of  Vtoi,  along  with  the  system  dynamics,  will  determine  the  voltage  ripple  Vrip  about  the  set-point 
Vref.  Typical  specifications  require  the  voltage  ripple  to  be  small,  so  that  the  output  voltage  Vout 
is  approximately  Vref,  that  is,  Vrip  is  chosen  so  that  for  Vout  =  Vref  ±  Vrip,  we  have  Vout  ~  Vref.  The 
sensor  model  performs  quantization  and  sampling,  as  would  occur  in  typical  analog  to  digital 
conversion  (ADC)  used  to  digitize  analog  signal  measurements.  The  actuator  models  likewise 
perform  the  inverse  process  of  digital  to  analog  conversion  (DAC)  to  convert  the  digital  (cyber) 
signals  to  analog  signals. 

Generally,  we  can  model  the  plant  as  a  physical  block,  the  hysteresis  controller  as  a  cyber  block, 
and  the  sensors  and  actuators  as  cyber-physical  blocks  in  SLSF.  The  plant  voltage  is  an  output 
physical  variable  that  affects  the  output  cyber  variable— a  switching  mode  signal  that  enables  the 
transition  between  each  mode  in  the  plant— of  the  controller,  and  vice-versa.  This  interaction 
between  the  plant  and  the  controller  is  accomplished  through  the  transitive  closure  of  input-output 
connections  with  the  sensor  and  the  actuator  in  the  SLSF  model.  We  will  formalize  specifications 
and  mismatches  of  the  buck  converter  in  Section  4.  As  a  prelude,  we  highlight  that  Hynger  finds 
its  candidate  invariant  (that  can  be  shown  to  be  an  actual  invariant  when  modeled  as  a  hybrid 
automaton  [22,  26,  40]). 

3.1  Cyber-Physical  Input-Output  Automata 

To  further  investigate  cyber-physical  specification  mismatches  of  CPS  models,  we  consider  ones 
that  may  be  formally  represented  as  cyber-physical  input-output  automata. 

Definition  3.5.  A  cyber-physical  input-output  automaton  (CPIOA)  VI  is  a  tuple,  lA  =  ( Loc ,  Var, 
Flow,  Inv,  Traj,  Lab,  Trans,  Init),  consisting  of  the  following  components: 

•  Loc:  a  finite  set  of  discrete  locations. 

•  Var:  a  finite  set  of  n  continuous,  real-valued  variables,  where  Vx  €  Var,  val(x)  e  R  and 
val(x)  is  a  valuation— a  function  mapping  x  to  a  point  in  its  type— here,  R;  and  Q  =  LocxR.” 
is  the  state  space.  Var  is  the  disjoint  of  a  set  of  input  variables  I  and  a  set  of  output  variables 
O.  Furthermore,  C  and  P  classify  Var  into  sets  of  cyber  and  physical  variables,  respectively. 

•  Inv:  a  finite  set  of  invariants  for  each  discrete  location,  VL  €  Loc,  Inv(L)  C  R”. 

•  Flow:  a  finite  set  of  derivatives  for  each  continuous  variable  x  6  Var,  and  Flow{{,  x)  C  R" 
describes  the  continuous  dynamics  of  each  location  €  €  Loc.  if  x  is  a  physical  variable, 
Flow(T,  x)  is  a  non-zero  Lipschitz  continuous  differential  equation  over  time.  Otherwise,  if 
x  is  a  cyber  variable,  Flow(£,  x)  =  0. 

•  Traj:  a  finite  set  of  continuous  trajectory  models  the  valuations  of  variables  over  an  interval 
of  real  time  [0,  T],  Let  Ao,  Af  and  Aj  be  the  valuations  of  variable  x  at  time  points  0,  t,  and 
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T  respectively,  Vf  e  [0,  T],  Vx  e  Var,  3£  e  Loc,  a  trajectory  r  e  Traj  is  a  mapping  function 
T  :  [0,  T]  — »  val(Var)  such  that: 
o  At  —  Ao  +  fg_0  Flow(£,x)d8,  and 
o  Ao  |=  Inv(£ ),  At  |=  Inv(£ ),  and  A j  |=  Inv(t). 

•  Lab :  a  finite  set  of  synchronization  labels. 

•  Trans :  a  finite  set  of  transitions  between  locations;  each  transition  is  a  tuple  y  =  (£,  £'  ,g,  u), 
which  can  be  taken  from  source  location  £  to  destination  location  £'  when  a  guard  condition 
g  is  satisfied,  and  the  post-state  is  updated  by  an  update  map  u. 

•  Init  is  an  initial  condition,  which  consists  of  a  set  of  locations  in  Loc  and  a  formula  over 
Var,  so  that  Init  C  Q. 

Next,  we  define  the  semantics  of  a  CPIOA  SR  in  term  of  executions.  An  execution  of  SR  is  a 
sequence  of  states,  written  as  p  —  so  — s >  Si  — s >  s2  ■  ■  •  ,  where  s o  G  Init,  and  s,-  — *  s!+i  is  the  update 
from  the  current-state  s;  to  the  post-state  s!+1,  that  is  specified  by  the  transition  relations  of  the 
CPIOA  SR  including:  (a)  a  discrete  transition  that  demonstrates  the  instantaneous  state  update,  or 
(b)  a  continuous  trajectory  that  represents  the  state  update  over  a  real  time  interval.  We  say  a  state 
Sk  is  reachable  from  an  initial  state  so  if  there  exists  an  execution  p  =  s$  — s >  Si  — s >  . . .  — s >  s^. 

Invariant  Property.  An  invariant  property  (p  of  a  CPIOA  SR  is  a  formula  over  Var  and  Loc  that 
is  always  true  for  every  reachable  state  of  SR.  Formally,  we  say  SR  |=  cp  iff  Vs  €  Reach(R),  s  |=  cp, 
where  Reach(R)  denotes  the  set  of  reachable  states  of  LR. 

Parallel  Composition.  Consider  two  CPIOAs  SR]  =  (Loci,  Var\,  Inv\,  F/owj,  Traj j,  Labi,  Trans\, 
Initi),  and  SR-i  =  ( L0C2 ,  Var2,  Inv 2,  Flo-w2,  Traj2,  Lab2,  Trans2,  Init2),  we  consider  that  SSR\  and  SR2 
is  compatible  if  (a)  C  02,  (b)  I2  £  Oj,  and  (c)  Oi  C\  O2  —  0.  The  parallel  composition  operation 
combines  two  compatible  CPIOAs  into  a  single  CPIOA  that  represents  the  synchronous  interaction 
between  these  two  CPIOA  when  running  simultaneously. 

Dejinition  3.6  (Parallel  Composition).  Given  two  compatible  CPIOAs  SRi  and  SR2,  the  parallel 
composition  of  SR\  and  SR2  is  a  CPIOA  SR  ,  written  as  SR  =  SR\ ||^2,  where: 

•  Loc  =  Loci  X  L0C2, 

•  Var  —  Vari  U  Var2, 

•  Q  =  QixQ2, 

•  O  —  Oi  U  O2, 

•  I  =  (h  U  h)  \  O, 

•  V£\,£2  g  Loc,  Inv(£i,t2)  =  Inv\(£\)  A  Inv2{£ 2) 

•  V£\,  £2  G  Loc,  Wx  e  Var,  ((£\,  £ 2 ),  val(x)  e  Init )  iff  (71?  val(x))  e  Initi  A  (£ 2 ,  val(x))  e  Init2, 

•  Lab  =  Labi  U  Lab2, 

•  Vi  G  {1, 2},  there  is  a  trajectory  r  G  Traj  iff  r  X  ( Loci  U  Varj)  €  Trajt,  where  r  J,  (Loc;  U  Var;) 
denotes  the  projection  of  r  onto  the  sets  of  variables  and  locations  of  component  i. 

•  Given  6  Transi,  yj  =  (£i,£[,gi,ui')  and  y2  G  Trans2,  y 2  =  (£2,  £^92,^2),  there  exists  a 
transition  y  6  Trans,  y  =  (£,£’ ,g,u)  iff: 

o  £  -  (£ 1 ,  £2),  £’  =  {£[,  £2),  g  =  gi,  and  u  -  ui,  or 

o  £  -  (£\,  £2),  £’  =  (£\,  £'2),  g  =  <72 >  and  u  =  u2,  or 

o  £  —  (£] ,  £2),  £'  —  (£[,  £'2),  g  —  gi  A  g2,  and  u  =  iq  U  u2. 

Closed-loop  CPIOA.  One  type  of  CPS  model  that  we  focus  on  in  this  paper  is  a  closed-loop  model, 

e.g.,  the  closed-loop  buck  converter.  Such  a  model  can  be  formally  represented  as  a  closed-loop 
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Fig.  3.  A  hybrid  automaton  models  the  buck  converter  plant  with  hysteresis  controller. 


CPIOA,  which  is  a  parallel  composition  of  a  plant  and  controller  CPIOA.  The  plant  CPIOA  has 
continuous  dynamics  modeled  by  ordinary  differential  equations,  and  the  controller  CPIOA  can  be 
purely  discrete.  For  instance,  the  hybrid  automaton  of  the  closed-loop  buck  converter  (without 
sensor  and  actuator)  shown  in  Figure  3  can  be  considered  as  one  closed-loop  CPIOA,  where  6  is  a 
synchronization  label  and  mode  is  a  discrete  control  signal.  The  capacitor  voltage  variable  Vc  is 
not  only  an  output  physical  variable  for  the  plant  CPIOA,  but  is  also  an  input  cyber  variable  of  the 
controller  CPIOA.  In  this  case,  we  can  check  whether  the  candidate  invariants  of  the  closed-loop 
buck  converter  found  with  Hynger  and  Daikon  are  actual  invariants  by  investigating  its  formal 
model  (e.g.,  a  closed-loop  CPIOA  shown  in  Figure  3)  using  some  hybrid  systems  model  checkers 
such  as  SpaceEx  [20]. 

3.2  Candidate  Invariant  Checking  Problem 

The  formal  definition  of  the  candidate  invariant  checking  problem  for  CPS  is  described  as  follows. 

Definition  3.7  (Candidate  Invariant  Checking).  Given  a  CPS  model  R  with  a  set  of  candidate 
invariants  <F,  R  is  a  formal  model  converted  from  R,  a  candidate  invariant  ip  6  O  is  considered  as 
an  actually  invariant  property  of  R  iff  Reach(R)  |=  ip. 

According  to  Definition  3.7,  if  a  CPS  model  R  is  a  white  box  system  that  can  be  represented  in 
terms  of  a  formal  model  such  as  a  CPIOA  R,  a  hybrid  system  model  checker  may  be  used  to  check 
whether  ip  is  an  invariant  property  of  R.  If  there  exists  any  reachable  state  of  R  that  does  not 
satisfy  ip,  we  can  conclude  that  ip  is  not  an  actual  invariant  of  the  CPS  model  R. 

4  CYBER-PHYSICAL  SPECIFICATIONS  AND  MISMATCHES 

In  this  section,  we  will  formalize  the  concept  of  candidate  cyber-physical  specification  mismatches 
of  CPS,  and  introduce  a  potential  method  to  detect  such  specification  mismatches. 
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4.1  Cyber-Physical  Specifications 

Our  goal  is  to  find  specifications  that  are  invariants  or  conditional  invariants,  so  we  do  not  consider 
more  general  temporal  logic  formulas.  Under  this  assumption,  a  specification  is  equivalent  to  a 
predicate  over  the  state-space  of  the  system.  Equivalently,  a  specification  is  a  multi-sorted  first-order 
logic  (FOL)  sentence  (of  a  restricted  class),  so  we  assume  the  specification  may  be  represented  in  the 
Satisfiability  Modulo  Theories  (SMT)  library  standard  language  [6,  35].  Under  these  assumptions, 
candidate  invariants  may  be  specified  as  quantifier-free  SMT  formulas  over  the  variables  of  the 
SLSF  model,  where  the  type  of  a  variable  corresponds  to  the  SMT  sort.  For  a  formula  (j),  let  vars(<f>) 
be  the  set  of  variables  appearing  in  <j>.  For  a  formula  (f>:  (a)  if  vars(fi)  are  all  physical,  then  f  is  a 
physical  specification,  (b)  if  vars(f>)  are  all  cyber,  then  0  is  a  cyber  specification,  and  (c)  if  vars(<f>) 
consists  of  both  cyber  and  physical  variables,  then  0  is  a  cyber-physical  specification. 

Next,  while  we  will  try  to  infer  interesting  specifications  (j)  using  dynamic  analysis  later  in  the 
paper,  we  first  highlight  examples  of  specifications  made  a  priori  in  system  design,  as  these  are 
necessary  to  define  specification  mismatches.  Fet  X  be  a  set  of  specifications  for  J?l,  which  is  a  set 
of  formulas  over  the  variables  of  3K.  Referring  to  Figure  4,  we  separate  the  specification  X  into 
sets  of  cyber  and  physical  specifications,  written  respectively  as  Xc  and  Xp.  These  specifications 
include  assumptions  about  the  physical  environment,  such  as  the  value  of  gravitational  force, 
temperature  bounds,  time  constants,  etc.  The  physical  specification  also  includes  assumptions 
about  the  physical  system’s  behavior  and  subcomponents,  such  as  motor  torque  limits,  temperature 
bounds  of  components,  sampling  rates,  velocity  limits,  etc.  Here  Xc  denotes  the  set  of  cyber 
specifications.  The  cyber  specifications  include  assumptions  about  software-physical  interfaces, 
such  as  ADC  resolution,  DAC  resolution,  sampling  rates,  etc.  It  also  includes  assumptions  about  the 
software  system,  subcomponents,  and  software-software  interfaces,  such  as  data  formats,  control 
flow,  event  orderings,  etc.  For  example,  the  buck  converter  has  the  following  physical  specifications: 

Up  —  t  >  ts  =>  Vouft)  —  Vref(f)  i  Vrip, 

4  =  Vs(t)  =  Vs( 0)  ±  8s, 

4  =  Vref(t)  =  Vref(0)  ±  Sref, 

and  Xp  =  {tip,  <jp,  <Tp}.  Here,  alp  states  that  after  some  amount  of  constant  startup  time  ts,  the 
output  of  the  buck  converter  Vout(t )  remains  near  a  reference  (desired)  output  voltage  Vref(t).  Both 
Up  and  Up  specify  assumptions  about  the  buck  converter’s  environment,  namely  that  its  source 
voltage  Vs  and  reference  voltage  Vref  always  remain  near  their  initial  values.  We  note  that  while 
time  may  not  typically  be  thought  of  as  a  state  of  the  system,  it  can  be  encoded  in  this  way  easily, 
for  example,  by  including  a  state  variable  t  with  t  —  1.  To  evaluate  whether  has  cyber-physical 
specification  mismatches,  we  hypothesize  that  the  cyber  specification  contains  (at  least  a  subset)  of 
the  physical  specification.  This  process  is  made  more  explicit  in  Figure  4  and  described  next. 

4.2  Cyber-Physical  Specification  Mismatches 

A  CPS  model  or  implementation  will  be  provided  as  an  SFSF  diagram,  denoted  Ch  as  formalized 
above.  Next,  LA  is  instrumented  using  the  Hynger  yielding  a  modified  SFSF  diagram  A.  Now,  A 
is  executed  to  generate  a  set  of  sampled,  finite-precision  traces  T  for  each  initial  condition  8  in  a 
set  of  initial  conditions  0,  which  effectively  corresponds  to  a  test  suite.  The  traces  T  are  analyzed 
using  dynamic  analysis  methods,  such  as  Daikon,  to  generate  a  set  of  candidate  invariants  <f>,  each 
element  q>  of  which  may  be  checked  as  actual  invariants  if  A  corresponds  to  a  formal  model  (e.g.,  a 
CPIOA)  or  may  be  converted  to  one,  A.  If  that  is  the  case,  then  a  hybrid  system  model  checker 
may  be  employed  to  see  if  (p  is  an  actual  invariant  (p,  and  the  set  of  actual  invariants  4>  is  collected. 
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Fig.  4.  Hynger  overview,  inference  of  physical  specifications  assumed  by  software,  and  cyber-physical 
specification  mismatch  identification. 

Definition  4.1  (Cyber-Physical  Specification  Mismatch).  Given  an  SLSF  diagram  LA  with  a  set  of 
actual  physical  specifications  Ep,  let  Op  =  €>  J,  Vsp  be  a  set  of  candidate  physical  invariant,  LA  has 
a  cyber-physical  specification  mismatch  iff:  3op  €  Ep,  Vi pp  €  Op,  op  (pp. 

In  Definition  4.1,  O  J,  Vsp  denotes  the  projection  or  the  restriction  of  O  to  the  set  of  software 
physical  variable  Vsp.  In  all  cases,  each  candidate  invariant  (p  €  O  is  projected  (restricted)  onto 
the  software  physical  variables  Vsp  to  yield  a  candidate  physical  invariant  (pp  and  corresponding 
set  Op.  Such  a  projection  may  be  computed  using  quantifier  elimination  methods  available  in 
many  modern  SMT  solvers,  such  as  Z3  [13] 2 .  Now,  Op  corresponds  to  the  candidate,  inferred 
physical  invariants  from  the  perspective  of  the  cyber-physical  system,  each  element  of  which  may 
be  compared  to  each  element  op  of  a  set  of  actual  physical  specifications  Ep.  Since  <pp  and  op 
are  both  formulas,  we  construct  new  formulas  (pp  =>  op  and  op  =>  i pp,  each  of  which  may  be 
discharged  with  an  SMT  solver.  If  these  checks  are  not  valid,  then  these  specifications  are  candidate 
cyber-physical  mismatches.  These  checks  basically  compare  whether  the  inferred  specification 
and  actual  specification  are  more  or  less  restrictive  than  one  another,  in  terms  of  the  sizes  of 
corresponding  sets  of  states  satisfying  the  predicates.  We  hypothesize  that  it  is  generally  the 
case  that  the  inferred  physical  specification  should  always  be  stronger  than  the  actual  physical 
specification,  and  only  the  check  i pp  =>  op  would  be  needed.  This  would  correspond  to  the  case 
where  the  software’s  assumptions  about  the  physical  world  are  at  least  as  restrictive  as  those  made 
in  the  actual  physical  specification.  For  instance,  suppose  that  the  physical  specification  of  the 
output  voltage  of  the  buck  converter  is  op  =  t  >  ts  =>  4.81^  <  Vout (t)  <  5.2V,  and  the  candidate 
physical  invariant  is  (pp  =  t  >  ts  =>  4.9V  <  Vout(t)  <  5.1V,  then  the  check  of  the  formula  <pp  =>  op 
using  an  SMT  solver  like  Z3  will  indicate  that  the  system  does  not  have  a  specification  mismatch. 
Otherwise,  if  the  candidate  physical  invariant  is  (pp  =  t  >  ts  =>  4.1V  <  Vout(t)  <  5. OF,  then  the 
check  of  the  formula  <pp  =>  op  will  indicate  that  the  system  has  a  specification  mismatch.  On 
the  other  hand,  it  may  also  be  useful  to  check  (pp  <=  op,  which  would  correspond  to  cases  where 
the  inferred  physical  specification  is  weaker  than  the  actual  physical  specification.  In  this  case, 
there  may  be  a  trace  that  violates  the  actual  specification,  and  this  may  be  useful  in  analysis  like 
falsification  to  drive  simulations  towards  a  violating  behavior. 

2Z3  may  be  downloaded:  http://z3.codeplex.com/. 
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5  HYNGER:  GENERATING  INVARIANTS  FOR  SLSF  MODELS 

Hynger— HYbrid  iNvariant  GEneratoR— is  a  software  tool  developed  for  invariant  inference  of  CPS 
models  represented  as  SLSF  block  diagrams3.  Hynger  is  written  primarily  in  Matlab  and  uses  the 
Matlab  APIs  to  interact  with  SLSF  diagrams.  Hynger  also  uses  some  Java  code  (natively  inside 
Matlab)  to  interface  with  Daikon,  which  is  written  in  Java.  Daikon  versions  5.0.0  to  5.1.8  were 
tested  with  Hynger4. 

Given  an  SLSF  model  J\,  Hynger  automatically  inserts  callback  functions  into  the  model  to 
print  model  variables  at  block  inputs  and  outputs  at  certain  events  in  the  SLSF  simulation  loop. 
Consequently,  a  trace  file  generated  by  Hynger  will  then  be  formatted  in  the  trace  input  format 
required  by  Daikon.  While  configurable,  the  default  behavior  of  Hynger  is  to  add  instrumentation 
(observation)  points  for  every  input  and  output  signal  for  every  block  (recursively)  in  the  SLSF 
diagram.  That  is,  Hynger  walks  the  tree  of  blocks  starting  from  the  root,  and  for  each  v  €  M, 
adds  instrumentation  points  for  the  input  variables  V j(v)  and  the  output  variables  V  oip)  of  v.  Of 
course,  this  may  incur  a  drastic  performance  overhead,  so  if  this  is  not  desired,  the  user  may  select 
only  a  subset  of  the  blocks  to  instrument  and  our  performance  results  (see  Section  6)  illustrate 
this  distinction.  When  an  SLSF  model  is  simulated  with  these  instrumentation  callback  functions 
added  by  Hynger,  it  will  generate  a  trace  file  in  the  input  trace  format  for  Daikon.  Hynger  also 
provides  the  capability  to  automatically  call  Daikon  from  Matlab  (by  using  an  appropriate  Java  call 
to  Daikon),  which  will  then  return  the  set  of  candidate  invariants  from  each  program  point  to  the 
user. 

The  Hynger  flow  is  summarized  in  Figure  4.  The  inputs  are:  (a)  SLSF  diagrams  (containing 
embedded  software  code  and  a  set  of  physical  variables  along  with  their  physical  dynamics  models 
[e.g.,  ODEs]),  and  ( b )  a  set  of  physical  variables  along  with  their  dynamics  models  (specified  as 
SLSF  children  diagrams),  and  (c)  a  test  suite  for  the  embedded  software  and  initial  conditions  for 
the  physical  simulation  (such  as  noisy  initial  conditions,  6  €  0).  The  output  of  the  Hynger  tool  is  a 
set  of  candidate  invariants,  which,  when  projected  onto  all  the  software  physical  variables  V$P , 
represent  a  candidate  specification  the  software  assumes  for  the  physical  parts  of  the  system.  Finally, 
candidate  specifications  can  be  checked  for  conformance  with  the  actual  physical  requirements 
by  comparing  the  two  specifications:  the  actual  physical  specification  and  the  candidate  physical 
specification  from  the  software  perspective. 

5.1  Dynamic  Invariant  Inference  with  Daikon 

Next,  we  illustrate  the  dynamic  invariant  inference  methodology  used  by  Daikon  on  a  pure  software 
example.  However,  this  pure  software  example  (a  C  function)  is  actually  specified  for  the  controller 
in  the  buck  converter  case  study  (shown  in  Figure  7)  in  a  different  manner.  The  loop  in  the  controller 
SLSF  model  of  Figure  9  also  computes  a  sum  of  an  array,  and  Daikon  can  find  this  specification  for 
both  the  SLSF  controller  model  using  Hynger,  and  the  C-frontend  for  the  following  example.  Note 
that,  in  Figure  9  the  digitized  output  voltage  from  the  buck-converter  plant  is  used  to  determine 
the  mode  of  the  switch.  Here,  Vtoi  is  denoted  by  the  variable  Vtol,  Vref  is  Vref .  We  highlight  that 
the  controller  computes  a  moving  average  by  summing  an  array.  With  Hynger  and  Daikon,  we 
automatically  infer  that  the  result  of  this  is  the  sum  of  the  samples,  similar  to  the  sum  return 
specification  shown  in  Figure  6  found  for  the  C  function  in  Figure  5. 

Example  C  Program,  Formal  Specification,  and  Candidate  Invariants  Inferred.  Figure  5  shows 
an  example  C  function  to  illustrate  the  use  of  dynamic  analysis  with  Daikon  to  find  candidate 

3  A  preliminary  prototype  of  Hynger  with  examples  is  available  online:  http://verivital.com/hynger/.  The  repository  also 
includes  Daikon  input  (*  .  dt  race)  trace  files  generated  from  the  examples,  as  well  as  the  Daikon  output  candidate  invariant 
(*  .  inv)  files. 

4Daikon  may  be  downloaded:  http://plse.cs.washington.edu/daikon/. 
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ensures  \ 
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\sum(0  ,n  —  1  ,\lambda  integer 

j:  b[j  ]); 

5 

@ 

ensures  \ 

result  >= 

0; 

//  false  ,  array  may  be 

negative 

*/ 

7 

int 

sum_array 

(int  b [ 1 , 

uns 

igned  int  n)  { 

int  i ; 

9 

int  s  =  0 

; 

/ *@  loop 

invariant 

11 

\  fora 

ll  int  ege 

r  j; 

(0  <=  i  <=  n)  ==>  s  == 

\sum  (0  ,  i  —  1  ,\  lambda  integer  j;  b[j]);  */ 

for  (i  = 

0;  i  <  n; 

i++ 

i 

13 

s 

+=  b [i] ; 

15 

} 

return  s ; 

Fig.  5.  Example  C  function  that  sums  an  array  b  of  n  integers.  Requirements  on  the  function  inputs  (i.e., 
preconditions  on  b  and  n  for  the  function  to  be  called)  are  specified  as  requires  assertions  in  the  ACSL 
language.  Correctness  specifications  (i.e.,  postconditions  following  the  function  call)  are  specified  as  ensures 
assertions  in  the  ACSL  language. 


==============  Precondition 

2  . . sum_array ( ) : : : ENTER 

b  has  only  one  value  //  it  's  a  pointer  to  only  one  location  of  memory 
4  b[]  elements  >=  0  //  all  elements  were  non-  negative  for  this  set  of  traces 

n  ==  100  //  all  tests  were  100  element  arrays  for  this  set  of  traces 
6  size(b[])  ==  100  //  all  tests  were  100  element  arrays 
==============  Postcondition 

8  . . sum_array ( ) : : : EXIT 

b[]  ==  orig(b[])  //  no  side  effects 
10  return  ==  sum(b[])  //  does  return  the  sum 
sum(b[])  ==  sum (orig (b [ ] ) ) 

12  b[]  elements  >=  0 


Fig.  6.  Daikon  candidate  invariant  output  (with  some  additional  markup  in  C-style  comments  for  readability) 
for  the  sum_array  example  from  Figure  5. 

invariants.  The  function  computes  and  returns  the  sum  of  an  array  of  integers.  This  example  was 
recreated  from  an  example  in  the  original  Daikon  paper  [17],  Additionally,  a  formalized  correctness 
specification  is  given  in  the  modern  ANSI/ISO  C  Specification  Language  (ACSL),  used  by  tools 
such  as  Frama-C  [12].  Using  Daikon  and  a  small  suite  of  unit  tests,  we  were  able  to  successfully 
find  the  invariant  that  returns  from  the  function  sum  array,  the  returned  value  is  the  sum  of  the 
elements  in  the  array  b.  The  suite  of  tests  included  arrays  with:  (a)  all  the  same  length  and  same 
elements,  ( b )  all  the  same  length  and  uniformly  randomly  chosen  elements,  (c)  different  lengths  and 
all  the  same  elements,  and  (d)  different  lengths  and  uniformly  randomly  chosen  elements.  Daikon 
successfully  found  the  sum  postcondition  in  all  these  cases  with  only  a  few  test  conditions.  The 
candidate  invariant  outputs  of  Daikon  appear  in  Figure  6,  where  we  can  see  Daikon  has  inferred  a 
candidate  invariant  that  the  function  returns  the  sum  of  an  array.  We  highlight  that  we  find  the 
sum  return  result  of  the  moving  average  filter  from  Figure  9  using  Hynger  and  Daikon. 

6  EXPERIMENTAL  RESULTS 

Hynger  was  tested  on  Windows  10  64-bit  using  Matlab  2016b,  and  2017a,  executed  on  a  x86-64 
laptop  with  a  2.3  GHz  dual-core  Intel  i5-6200U  processor  and  12  GB  RAM.  All  performance  metrics 
reported  were  recorded  on  this  system  using  Matlab  2017a.  We  tested  and  evaluated  Hynger  using  a 
number  of  SLSF  examples,  including:  (a)  the  closed-loop  buck  converter  with  sensor  and  hysteresis 
controller  described  in  Section  6.1  and  detailed  further  in  [40],  (b)  a  solar  array  case  study  that  uses  a 
buck-boost  converter  [42],  (c)  benchmarks  from  S-TaLiRo  [4],  (d)  benchmarks  from  Breach  [14,  24], 
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Model 

Solver 

Tmax 

Sim 

Simlnst 

Inv 

Overhead 

BDAII 

BDInst 

BDPct 

buck  (Section  6.1) 

ode45 

0.0083 

6.2985 

38.4518 

5.7335 

7.0152 

14 

3 

21.4286 

buck  (Section  6.1) 

ode45 

0.0083 

6.4567 

44.698 

7.0913 

8.021 

14 

4 

28.5714 

buck  (Section  6.1) 

ode45 

0.0083 

6.5301 

78.3176 

7.2224 

13.0993 

14 

14 

100 

heat25830  [4] 

ode45 

50 

4.6913 

254.5776 

14.09 

57.2692 

28 

1 

3.5714 

heat25830  [4] 

ode45 

50 

4.7328 

2882.7808 

15.6488 

612.4233 

28 

10 

35.7143 

fuell  [23] 

odel5s 

15 

5.3747 

976.6274 

7.923 

183.182 

208 

17 

8.1731 

fuell  [23] 

odel5s 

15 

4.2131 

2824.2804 

11.604 

673.1137 

208 

63 

30.2885 

fuel2  [23] 

odel5s 

20 

3.3838 

36.8312 

2.9881 

11.7674 

25 

6 

24 

fuel2  [23] 

odel5s 

20 

2.7353 

42.4074 

3.2771 

16.7018 

25 

13 

52 

fuel3  [19] 

odel5s 

20 

3.7425 

292.9976 

4.1131 

79.3892 

90 

11 

12.2222 

fuel3  [19] 

ode 15s 

20 

3.6083 

945.3992 

4.3904 

263.2236 

90 

46 

51.1111 

Table  1.  Hynger  performance  results  for  several  of  the  examples  evaluated.  Solver  is  the  ODE  solver  used  by 
SLSF.  Tmax  is  the  virtual  simulation  time  in  seconds  (i.e.,  time  from  the  perspective  of  the  model).  All  runtime 
results  are  in  seconds  and  are  the  mean  of  20  runs.  Sim  is  the  simulation  runtime  (s).  Inv  is  the  invariant 
generation  runtime  (Daikon)  (s).  Overhead  is  the  overall  relative  performance  overhead  (extra  runtime) 
(x)  using  Hynger  and  Daikon  versus  only  SLSF  simulation  (i.e.,  (( Simlnst  +  Inv)/ Sim)).  BDInst  and  BDAII 
are  the  numbers  of  block  diagrams  instrumented  and  the  overall  number  of  block  diagrams,  respectively. 
BDPct  is  the  percentage  (%)  of  block  diagrams  instrumented  using  different  Hynger  modes  of  operation  (i.e., 
BDInst /BDAII). 

(e)  benchmarks  created  as  a  part  of  the  ARCH  2014  CPSWeek  workshop  (particularly  [23,  40])  and 

(f)  example  models  provided  by  Mathworks.  Overall,  these  examples  vary  from  fairly  simple  with 
tens  of  blocks  (such  as  the  buck  converter  case  study  we  detail),  to  complex  (with  hundreds  of 
blocks). 

Runtime  Overhead  from  Instrumentation  with  Hynger  and  Invariant  Inference  with  Daikon.  First, 
we  present  an  aggregate  performance  evaluation  for  some  of  these  examples  in  Table  1,  with 
column  descriptions  appearing  in  the  caption.  Overall,  the  performance  overhead  of  instrumenting 
diagrams  and  performing  invariant  inference  is  around  an  order  of  magnitude  increase  in  the 
best  cases,  and  two-to-three  orders  of  magnitude  increase  in  the  worst  cases,  which  we  note  is 
comparable  with  typical  Daikon  instrumentation  frontends  like  Valgrind’s  overhead  [18,  39].  We 
conducted  performance  profiling  of  Hynger  and  identified  the  main  source  of  overhead  (about  75  to 
90  percent)  as  file  I/O  operations.  Additionally,  as  Hynger  has  several  different  usage  scenarios  and 
operating  modes  (where  it  may  be  used  to  instrument  few  blocks  [subsystem  and  function  blocks 
by  default],  many  blocks  [all  blocks  except  ones  such  as  constants,  scopes,  etc.],  every  single  block, 
or  user-selected  blocks),  the  table  illustrates  these  differences  to  give  some  comparison  of  how  the 
methods  scale  on  a  given  model.  Next,  we  will  describe  two  CPS  case  studies  in  details  to  evaluate 
the  capability  of  Hynger  in  detecting  cyber-physical  specification  mismatches.  The  first  model 
is  the  closed-loop  buck  converter  that  has  been  used  to  illustrate  the  concepts  of  this  paper,  and 
the  second  model  is  derived  from  a  collection  of  the  automotive  powertrain  control  benchmarks 
proposed  by  Toyota  [24]. 

6.1  Closed-Loop  Buck  Converter  Cyber-Physical  Specification  Mismatch 

A  basic  cyber-physical  specification  mismatch  is  easy  to  encode  in  the  buck  converter,  since  the 
software  controller  inherently  uses  a  tolerance  to  encode  the  desired  output  voltage  ripple.  This 
hysteresis  tolerance  band  is  typically  chosen  based  on  the  system  dynamics  and  desired  output 
voltage  ripple  to  ensure  the  output  voltage  meets  the  ripple  specification.  As  a  concrete  example, 
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DC-to-DC  Converter  Plant 


Fig.  7.  General  CPS  case  study  architecture  overview  of  the  buck  converter  in  SLSF.  The  system  is  composed 
of  a  plant  (physical  system)  model,  a  controller  (software/cyber),  and  potentially  sensor  and  actuator  models. 
The  cyber  model  uses  some  of  the  physical  model  output  states  to  determine  a  control  action  or  input. 
The  controller  in  SLSF  appears  in  Figure  9,  and  the  sensor  model  appears  in  Figure  8.  An  example  of  this 
closed-loop  buck  converter  including  only  plant  and  controller  can  be  formally  represented  as  the  hybrid 
automaton  in  Figure  3. 

the  physical  specification  may  contain  a  fixed  constraint  that  Vout  —  Vref  ±  Vrip,  e.g.,  Vref  —  5V  and 
Vrjp  —  0.117.  The  hysteresis  band  Vtoi  is  then  selected  based  on  the  system  dynamics  to  ensure 
4.9V  <  yout  <  5.117  so  that  it  meets  the  requirements  of  the  physical  specifications  defined  by  Ep 
in  Section  4.1. 

Sources  of  Cyber-Physical  Specification  Mismatches  of  the  Closed-Loop  Buck  Converter.  There  are 
different  possibilities  of  specification  mismatch  that  may  occur  to  the  closed-loop  buck  converter. 
We  present  three  scenarios  that  result  in  specification  mismatches.  First,  if  the  plant  parameters 
change  (i.e.,  different  circuit  elements  are  used),  and  the  software  is  not  updated  with  a  new 
hysteresis  band  Vto;  to  accommodate  the  changes  in  the  plant  dynamics,  then  a  specification 
mismatch  manifests.  This  mismatch  can  be  detected  using  Hynger  and  the  methodology  described 
in  this  paper.  Of  course,  this  is  a  somewhat  obvious  mismatch,  as  the  controller  relies  on  variables 
computed  as  functions  of  the  plant  parameters  (here,  the  R,  L,  and  C  values,  as  well  as  the  source 
and  desired/reference  output  voltage  values).  So  if  these  plant  components  are  changed,  clearly 
the  software  must  be  updated.  Second,  the  hysteresis  controller  is  initially  constructed  using 
wrong  information  about  the  physical  evolution  of  the  plant.  In  fact,  the  hysteresis  band  1 7toi  is  far 
different  from  the  actual  output  voltage  ripples  Vrip  of  the  plant.  Third,  the  analog  sensor  of  the  buck 
converter  may  have  ADC  conversion  errors  that  reduce  the  accuracy  of  the  voltage  measurement. 
These  errors  can  be  an  offset  error,  a  full-scale  error,  differential  and  integral  non-linearity  errors, 
etc.  Moreover,  a  typical  error  that  cannot  be  avoided  in  ADC  sensor  is  the  quantization  error  [50]. 
Overall,  these  conversion  errors  may  cause  a  significant  impact  to  result  in  system  failures. 

Experimental  Results  in  Identifying  Cyber-Physical  Specification  Mismatches  of  the  Closed-Loop 
Buck  Converter.  We  consider  the  closed-loop  buck  converter  LA  shown  in  Figure  7  with  V5  =  100, 
Vref  =  4817,  Vrip  —  5%Vref  =  2.4V,  and  assume  that  Ss,  8ref  are  equal  to  zero.  The  physical 
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I  I  restart  circular 
//  buffer 
{index  =  0;} 


Fig.  8.  Stateflow  model  of  sensor  with  a  sample  Fig.  9.  Stateflow  model  of  the  buck-converter 

and  hold  for  the  buck  converter  case  study.  voltage  hysteresis  controller. 


specification  of  the  output  voltage  is  up  =  t  >  ts  =>  45.6V  <  Vout(t)  <  50.4V.  For  the  initial  setup, 
with  R  —  6fl,  L  =  2.65 mH,  C  —  2.2 mF,  and  a  sampling  frequency  fs  —  60 kHz,  the  magnitude  bound 
of  the  output  voltage  inferred  from  Hynger  and  Daikon  is  fp=  t  >  ts  =>  46.559V  <  Vout(t)  < 
50.203V.  Then,  fpp  is  considered  as  the  candidate  invariant  of  the  system  since  the  formula  tpp  =>  up 
is  true.  Next,  we  investigate  different  possibilities  of  cyber-physical  specification  mismatches  that 
may  occur  when  changing  the  source  voltage,  the  desired/reference  output  voltage,  the  sampling 
frequency,  and  the  plant  parameters  of  the  buck  converter. 

First,  we  increase  the  source  voltage  Vs  from  100V  to  120V,  the  new  magnitude  bound  of 
the  output  voltage  inferred  from  Hynger  and  Daikon  is  ipp  =  t  >  ts  =>  46.804V  <  Vout(t)  < 
51.118  V.  Then,  the  formula  <pp  =>  <xp  is  false,  that  indicates  the  system  may  have  a  cyber-physical 
specification  mismatch. 

Second,  we  drop  the  desired/reference  output  voltage  Vref  to  36  V.  Thus,  the  physical  specification 
of  the  output  voltage  becomes  a'p  =  t  >  ts  =>  34.2  V  <  Vout(t)  <  37.8V.  In  this  case,  the  inferred 

physical  specification  of  the  output  voltage  from  Hynger  and  Daikon  becomes  ipp  =  t  >  ts  => 
35.068V  <  Vout{t)  <  39.053V,  so  that  the  formula  (p'p  =>  trp  is  false.  Therefore,  changing  the 
reference  output  voltage  may  also  produce  a  cyber-physical  specification  mismatch  for  the  buck 
converter. 

Third,  we  decrease  the  sampling  frequency  fs  from  60 kHz  to  30 kHz.  As  a  result,  the  new  inferred 
physical  specification  of  the  output  voltage  from  Hynger  and  Daikon  is  <pp  =  t  >  ts  =>  45.853V  < 
Voutit)  <  51.091V.  The  check  of  the  formula  ipp  =>  up  will  return  false  to  indicate  that  the  system 
may  contain  a  cyber-physical  specification  mismatch. 

Next,  we  keep  the  controller  unchanged  and  vary  the  values  of  R,  L,  and  C  to  change  the  plant 
parameters.  We  then  run  the  buck  converter  with  Hynger  in  conjunction  with  Daikon,  and  collect 
candidate  physical  specifications  associated  with  the  output  voltage.  The  comparison  between 
the  actual  physical  specification  up  and  the  physical  specification  cpp  inferred  from  Hynger  and 
Daikon  is  shown  in  Table  2,  and  also  illustrated  in  Figure  10.  Note  that  in  Table  2,  (pp  describes 
the  magnitude  bound  of  the  output  voltage  when  t  >  ts.  The  checks  of  the  formula  (pp  =>  up 
occasionally  return  False,  that  are  depicted  in  Figure  10  when  the  bound  of  the  inferred  output 
voltage  overlaps  its  actual  bound.  This  indicates  that  changing  the  plant  parameters  without 
updating  the  controller  may  produce  cyber-physical  specification  mismatches.  That  also  proves 
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Parameter  Values 

VP 

<pp  =>  ap 

ap  =>  i ftp 

R  —  4Q,  L  =  2.65 mH,  C  =  2.2 mF 

45.137V  <  Vout(t)  <  49.723V 

False 

False 

R  —  8fl,  L  =  2.65 mH,  C  =  2.2 mF 

46.964V  <  Vout(t)  <  50.405V 

False 

False 

R  =  6Q,  L  =  0.65 mH,  C  =  2.2 mF 

47.141V  <  Vout(t )  <  50.074V 

True 

False 

R  =  6Q,  L  =  6.65 mH,  C  =  2.2 mF 

45.429V  <  Vout{t)  <  50.439V 

False 

True 

R  =  6Q,  L  =  2.65 mH,  C  =  1.2 mF 

45.426V  <  V0ut(t)  <  51.109V 

False 

True 

R  =  6Q,  L  =  2.65 mH,  C  =  3.2 mF 

46.859V  <  Vout(t)  <  49.774V 

True 

False 

Table  2.  Experimental  data  showing  the  comparison  between  actual  physical  specifications  and  inferred 
physical  invariants  from  Hynger  and  Daikon  of  the  buck  converter  system.  Here,  the  plant  component  is 
changed  due  to  the  changes  of  R,  L ,  and  C  values. 


the  capability  of  Hynger  and  our  proposed  methodology  in  automatically  detecting  a  candidate 
cyber-physical  specification  mismatch  of  CPS. 

Another  possibility  of  the  specification  mismatch  may  occur  when  the  controller  is  encoded 
based  on  wrong  information  about  the  plant.  For  the  buck  converter,  the  hysteresis  controller  is 
built  with  an  assumption  that  the  output  voltage  ripple  Vrip  is  equal  to  5%  of  the  reference  voltage 
Vref.  However,  the  actual  value  of  Vrip  may  be  much  smaller  than  this  assumption  percentage.  The 
percentage  of  the  output  voltage  ripple  of  the  buck  converter  is  calculated  as  follows  [16], 


Vnp  1  —  D 
\Q  ~  8LC ’ 


(1) 


where  D  -  is  a  duty  cycle,  and  r]  is  an  efficiency  coefficient  of  the  converter.  Here,  with 
L  =  2.65 mH,  C  —  2.2 mF,  fs  =  60 kHz,  rj  =  0.79,  Vref  =  48V,  and  'Vs  =  100 V,  the  percentage  of  the 
output  voltage  ripple  is  approximately  equal  to  0.0002%.  Thus,  the  hypothesized  output  voltage 
ripple  used  to  build  the  controller  is  far  larger  than  the  actual  output  voltage  ripple  calculated 
by  Equation  1.  It  definitely  shows  that  the  system  may  have  specification  mismatches  since  the 
controller  is  encoded  depending  on  wrong  information  about  the  physical  plant. 

Furthermore,  changing  the  length  of  voltage  measurement  array  (samples_length)  in  the  sensor 
of  the  buck  converter  (shown  in  Figure  8)  may  also  cause  a  specification  mismatch.  For  example,  if 
we  increase  it  from  16  to  32,  the  inferred  physical  specification  using  Hynger  and  Daikon  becomes 
i pp  =  t  >  ts  =>  46.095V  <  Vout(t )  <  50.788V,  which  no  longer  implies  the  actual  physical 
specification  of  the  output  voltage  <jp  =  t  >  ts  =>  45.6  V  <  Vout(t)  <  50.4V. 


6.2  Abstract  Fuel  Control  System  Benchmarks 

In  the  second  case  study,  we  present  the  potential  cyber-physical  specification  mismatches  of  the 
abstract  fuel  control  (AFC)  system  benchmarks  provided  by  Toyota  [23,  24],  and  further  studied 
in  [19].  The  goal  of  these  benchmarks  is  to  determine  the  fuel  rate  that  should  be  injected  into 
the  manifold  to  maintain  the  air-fuel  ratio  within  a  desirable  range  using  the  feedforward  and 
Proportional-Integral  (PI)  controllers.  Particularly,  we  focus  on  the  third  model  of  the  benchmarks 
including  a  sequence  of  Simulink  blocks  and  Stateflow  chart  that  increase  levels  of  sophistication 
and  fidelity  of  the  system  [19].  The  model  consists  of  four  operation  modes  and  four  continuous 
variables.  The  modes  include  startup,  normal,  power,  and  failure-,  and  the  variables  are  (a)  p\  an 
intake  manifold  pressure,  (b)  pe\  an  intake  manifold  pressure  estimate,  (c)  A:  an  air-fuel  ratio,  and 
( d )  i:  an  integrator  state,  PI  control  signal.  The  evolution  of  the  continuous  variables  in  each  mode 
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Fig.  10.  A  plot  represents  simulation  traces  and  magnitude  bounds  of  Vout  of  the  buck  converter  with  different 
values  of  R,  L,  and  C.  Here,  rxp  denotes  the  actual  bound  of  Vout ,  and  (ftp,  k  6  [1,  6]  denotes  the  inferred  bound 


of  Vout  listed  orderly  in  Table  2. 

is  governed  by  nonlinear  polynomial  differential  equations  as  follows, 

P  =  Ci(20(c2Op2  +  c2lp  +  c22)  -  mc)  (2) 

Pe  =  Ci(2C23<9(C20p2  +  C2ip  +  C22)  ~  (c2  +  C200pe  +  C^COp2  +  CsOi'pe))  (3) 

A  =  c26(ci5  +  (ci6c25fc  +  cvc22iF2c  +  c18mc  +  c19rhcc25Fc  -  A)  (4) 

i  =  Ci4(c24A  -  cn),  (5) 


where  Fc  =  ^-(1  +  i  +  Ci3(c24A-  cn))(c2  +  c2o)pe  +do)pl  +C5tu2pe),  and  mc  =  Ct2(c2  +  c^utp  +  c^cop2  + 
csO)2p).  9  and  to  are  throttle  angle  (in  degrees)  and  engine  speed  inputs  (in  rpm ),  respectively.  The 
values  of  all  constant  parameters  Cj,j  €  [1,25],  8  and  to  are  specified  in  [24].  We  note  that  this 
system  can  be  formally  represented  as  a  closed-loop  CPIOA,  which  is  the  parallel  composition  of 
a  plant  and  controller  model,  and  both  of  them  have  three  exogenous  inputs  including  9,  to,  and 
sensor  failure  event  fail_event  [19]. 

AFC  Plant  Model.  The  plant  can  be  modeled  as  a  CPIOA  with  a  single  mode  and  two  output 
physical  variables  p,  A  whose  continuous  evolutions  over  time  are  described  in  Equation  2  and 
Equation  4,  respectively.  This  model  has  an  input  cyber  variable  Fc,  that  is  a  fuel  command. 

AFC  Controller  Model.  The  controller  model  is  a  CPIOA  with  four  operation  modes  including 
startup ,  normal,  power,  and  failure.  The  controller  has  two  output  physical  variablespe,  and  i  whose 
continuous  evolutions  over  time  are  described  in  Equation  3  and  Equation  5,  respectively.  Here,  p 
and  A  are  considered  as  two  input  cyber  variables  of  the  controller. 

Reachability  analysis  of  a  sophisticated  system  like  the  AFC  system  is  a  major  contribution  to 
both  industrial  and  research  community.  However,  it  is  a  challenge  to  design  and  verify  such  a 
system  using  existing  hybrid  system  verification  tools.  Instead,  we  can  attempt  to  verify  some 
safety  requirements  of  the  system.  The  AFC  system  has  several  actual  physical  specifications  that 
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can  be  found  in  [15].  In  this  section,  we  select  two  main  physical  specifications  to  evaluate  the 
capability  of  Hynger  and  the  proposed  methodology.  The  first  physical  specification  requires  the 
undershoot  and  overshoot  of  the  air-fuel  ratio  of  the  system  should  be  in  the  settling  region  of  +2% 
of  its  reference  value  Arey.  The  second  physical  specification  requires  the  air-fuel  ratio  should  be 
maintained  within  +2%  of  Arep  in  the  normal  mode  when  t  >  ts.  These  properties  can  be  formally 
expressed  as: 


Up  =  mode  —  startup  A  t  <  ts  =>  0.98 Aref  <  A(f)  <  1.02Arey  (6) 

Up  =  mode  =  normal  A  t  >  ts  =>  0.98 Aref  <  A(f)  <  1.02 Arey.  (7) 

Initially,  we  set  Aref  =  14.7,  9  e  [8.8°,  90°],  w  =  1800rpm  ts  =  9.5s,  and  the  maximum  simulation 
time  Tmax  =  20s,  the  proportional  and  integral  gains  of  the  PI  controller  are  C13  =  0.04  and 
C14  =  0.14,  respectively.  Next,  we  investigate  different  possibilities  of  cyber-physical  specification 
mismatches  for  each  physical  specification.  For  the  first  physical  specification  ap,  the  AFC  system 
may  have  specification  mismatches  when  changing  the  engine  speed  and  throttle  inputs.  For  the 
second  physical  specification  a the  system  may  contain  specification  mismatches  when  changing 
controller  and  plant  parameters. 

Cyber-physical  specification  mismatches  according  to  oj,.  With  the  initial  setup  mentioned  earlier, 

the  physical  specification  in  Equation  6  becomes  up  =  mode  —  startup  A  t  <  9.5  =>  14.406  < 
A(t)  <  14.994.  Here,  the  magnitude  bound  of  the  air-fuel  ratio  at  the  startup  mode  of  the  system 
inferred  from  Hynger  and  Daikon  is  <f>p  =  mode  —  startup  At  <  9.5  =>  14.505  <  A(f)  <  14.97. 
Thus,  the  check  of  the  formula  cpp  =>  oj,  is  valid,  that  indicates  (ftp  is  a  candidate  invariant  of  the 
AFC  system.  Next,  we  vary  the  input  values  and  observe  the  consequent  behaviors  of  the  system. 

First,  we  vary  the  value  of  the  engine  speed  and  keep  other  parameters  unchanged.  Assuming 
•w  —  2200 rpm,  the  inferred  physical  specification  of  the  air-fuel  ratio  from  Hynger  and  Daikon 
becomes  <j)p  =  mode  =  startup  A  t  <  9.5  =>  14.129  <  Vout(t)  <  15.033.  Hence,  the  formula 
1 pp  =>  Up  is  false  indicating  that  the  AFC  system  may  contain  a  cyber-physical  specification 
mismatch  as  we  change  the  engine  speed  input. 

Second,  we  change  the  range  of  the  throttle  input  to  [40°,  70°].  Then,  the  inferred  physical 
specification  of  the  air-fuel  ratio  from  Hynger  and  Daikon  becomes  <pp  =  mode  =  startup  A  t  < 
9.5  =>  14.396  <  Vout(t)  <  14.849.  Hence,  (j>lp  no  longer  implies  tjp.  Therefore,  there  exists  a 
cyber-physical  specification  mismatch  when  changing  the  throttle  input  as  well. 

Cyber-physical  specification  mismatches  according  to  <jp.  Initially,  the  physical  specification  in 
Equation  7  is  Up  =  mode  =  normal  At  >  9.5  =>  14.406  <  A(t)  <  14.994.  Here,  the  magnitude 
bound  of  the  air-fuel  ratio  at  the  normal  mode  of  the  system  inferred  from  Hynger  and  Daikon  is 
1 p2p  =  mode  =  normal  A  t  >  9.5  =>  14.645  <  Aft)  <  14.84.  Then,  we  can  consider  < p2p  as  a  candidate 
invariant  of  the  system  because  the  formula  <pp  =>  0 p  is  true. 

Next,  we  investigate  whether  there  is  a  specification  mismatch  for  the  AFC  system  as  we  change 
the  proportional  and  integral  gains  of  its  PI  controller.  Table  3  describes  the  comparison  between  the 
actual  physical  specification  Op  and  the  physical  specification  (p2p  inferred  from  Hynger  and  Daikon, 
where  tp2p  J,  A  denotes  the  inferred  bound  for  A  when  t  >  ts  and  mode  —  normal.  In  Table  3,  the  check 
of  the  formula  < p2p  =>  a p  returns  false  in  some  cases  (e.g.,  when  C13  =  0.04,  c14  =  0.04)  indicating 
that  the  changes  in  the  controller  gains  may  produce  cyber-physical  specification  mismatches  for 
the  AFC  system. 
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Controller  Gain 

<P2p 

(P  p  =>  Gp 

Op  =>  (pp 

C13  =  0.01,  C14  =  0.14 

14.567  <  A(f)  <  15.058 

False 

False 

C13  =  0.02,  c\4  —  0.14 

14.592  <  A(f)  <  15.033 

False 

False 

C\ 3  —  0.06,  C\ 4  =  0.14 

14.634  <  A(t)  <  14.955 

True 

False 

C\ 3  —  0.8,  C\ 4  =  0.14 

14.642  <  A(t)  <  14.929 

True 

False 

Cj3  =  0.04,  ci4  =  0.04 

14.649  <  A(t)  <  15.007 

False 

False 

C13  =  0.04,  Cj4  =  0.34 

14.581  <  A(f)  <  14.937 

True 

False 

C13  =  0.04,  ci4  =  0.64 

14.577  <  A(f)  <  14.888 

True 

False 

C13  =  0.04,  Cj4  =  0.94 

14.589  <  A(t)  <  14.855 

True 

False 

Table  3.  Experiment  results  illustrate  the  comparison  between  actual  physical  specifications  and  inferred 
physical  invariants  from  Hynger  and  Daikon  of  the  AFC  system  when  changing  the  proportional  gain  and 
the  integral  gain  of  its  PI  controller. 

7  DISCUSSION 

Identifying  a  cyber-physical  specification  mismatch  of  CPS  with  dynamic  analysis  is  a  challenging 
problem.  Although  the  Hynger  prototype  in  conjunction  with  Daikon  can  detect  potential  cyber¬ 
physical  specification  mismatches  of  CPS,  such  as  those  in  the  case  studies  described  in  Section  6, 
however,  it  has  some  limitations.  First,  the  Daikon  tool  used  by  Hynger  may  only  infer  extremely 
limited  classes  of  nonlinear  invariants  by  default  (e.g.,  squares  like  x2),  and  not  general  polynomials 
(e.g.,  x2  +  y2  +z3).  So  we  plan  to  extend  the  invariant  templates  to  be  able  to  capture  more  interesting 
relations,  particularly  for  physical  variables.  Second,  although  Daikon  can  infer  candidate  invariants 
in  terms  of  logical  predicates  over  variables,  it  has  limitation  for  checking  complex  specifications 
related  to  real-time  requirements  such  as  STL,  MTL  and  HyperSTL  [41].  Industrial-scale  CPS 
usually  have  safety  and  liveness  requirements  depending  on  precise  real-time  relations  of  signals, 
so  strengthening  the  capability  of  checking  temporal  logic  like  STL,  MTL  and  HyperSTL  in  Daikon 
would  leverage  the  methodology  presented  in  this  paper. 

Additionally,  while  the  Hynger  tool  is  a  prototype,  it  can  be  envisioned  to  take  an  arbitrary  SLSF 
model,  instrument  it,  feed  the  resulting  traces  to  Daikon  to  generate  candidate  invariants,  then 
check  if  these  candidate  invariants  are  actual  invariants  or  not  (using,  e.g.,  SpaceEx  [20]  or  other 
hybrid  system  model  checkers),  as  well  as  identify  specification  mismatches.  For  example,  the 
candidate  invariants  inferred  from  Hynger  and  Daikon  of  the  buck  converter  including  only  plant 
and  controller  represented  in  term  of  hybrid  automata  in  Figure  3  would  easily  be  checked  to  see 
whether  they  are  actually  invariants  using  SpaceEx.  In  long  term,  Hynger  could  be  extended  for 
runtime  assurance  tasks  like  detecting  and  thwarting  security  violations  and  attacks,  similar  to  the 
ClearView  tool  that  also  uses  Daikon  [47],  ClearView’s  success  for  software  systems  illustrates  that 
finding  sets  of  candidate  invariants  and  monitoring  their  evolution  over  time  may  be  useful  for 
runtime  assurance  and  resiliency  methods  in  CPS.  If  the  candidate  invariants  are  checked  at  runtime 
using  a  real-time  reachability  method  [5],  a  formal  and  dynamic  runtime  assurance  environment 
may  be  feasible. 

8  CONCLUSION  &  FUTURE  WORKS 

The  results  illustrate  the  feasibility  of  using  dynamic  invariant  inference  for  analysis  of  embedded 
and  cyber-physical  systems.  The  Hynger  prototype  enables  a  powerful  extension  of  dynamic 
invariant  inference  to  CPS  for  two  main  reasons.  First,  it  enables  potentially  model-free  and  black 
box  invariant  inference,  since  the  internals  of  the  SLSF  blocks  may  remain  unknown.  If  no  model 

,  Vol.  1,  No.  1,  Article  1.  Publication  date:  January  201X. 

APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

214 


Cyber-Physical  Specification  Mismatches 


1:23 


is  available  (in  the  black  box  case),  the  candidate  invariants  represent  what  may  be  the  most  formal 
model  available,  otherwise  (in  the  white  box  case),  then  candidate  invariants  represent  a  candidate 
abstraction  of  that  model.  If  the  candidate  invariants  are  actual  invariants,  this  is  powerful,  as  they 
represent  what  is  likely  a  less  complex  representation  of  the  set  of  reachable  states  of  the  system. 
Second,  if  we  view  the  SLSF  models  as  hybrid  automata  in  a  formal  context,  it  represents  the  first 
use  of  dynamic  execution  analysis  for  hybrid  systems  with  sophisticated  software  state  and  discrete 
complexity.  Two  proof-of-concept  CPS  case  studies  including  the  DC-to-DC  power  converter  and 
the  powertrain  fuel  control  system  are  presented  to  illustrate  the  capability  of  Hynger  in  detecting 
potential  cyber-physical  specification  mismatches. 

Overall,  there  are  several  directions  for  future  research,  including:  (a)  extending  the  classes  of 
invariants  that  may  be  inferred,  particularly  to  nonlinear  (polynomial)  [43]  and  disjunctive/max-plus 
forms  [45],  potentially  by  integrating  Daikon  with  techniques  from  Dig  [44],  ( b )  runtime  assurance 
and  verification  with  real-time  reachability  of  inferred  invariants  [5],  (c)  improving  and  refining 
Hynger,  particularly  with  regard  to  performance  (such  as  using  Daikon  in  the  online  mode  with 
direct  pipes  between  Hynger  and  Daikon,  so  that  file  I/O  is  minimized),  and  ( d )  analyzing  more 
industrial-scale  CPS  using  Hynger. 
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Abstract.  The  objective  of  NASA’s  Small  Aircraft  Transportation  Sys¬ 
tem  (SATS)  Concept  of  Operations  (ConOps)  is  to  facilitate  High  Vol¬ 
ume  Operation  (HVO)  of  advanced  small  aircraft  operating  in  non- 
towered  non-radar  airports.  Given  the  safety-critical  nature  of  SATS, 
its  analysis  accuracy  is  extremely  important.  However,  the  commonly 
used  analysis  techniques,  like  simulation  and  traditional  model  checking, 
do  not  ascertain  a  complete  verification  of  SATS  due  to  the  wide  range 
of  possibilities  involved  in  SATS  or  the  inability  to  capture  the  ran¬ 
domized  and  unpredictable  aspects  of  the  SATS  ConOps  environment 
in  their  models.  To  overcome  these  limitations,  we  propose  to  formulate 
the  SATS  ConOps  as  a  fully  synchronous  and  probabilistic  model,  i.e. , 
SATS-SMA,  that  supports  simultaneously  moving  aircraft.  The  distin¬ 
guishing  features  of  our  work  include  the  preservation  of  safety  of  aircraft 
while  improving  throughput  at  the  airport.  Important  insights  related 
to  take-off  and  landing  operations  during  the  Instrument  Meteorological 
Conditions  (IMC)  are  also  presented. 

Keywords:  Formal  Verification,  Probabilistic  Analysis,  Model  Check¬ 
ing,  SATS,  SATS  Concept  of  Operations,  Aircraft  Safety,  Aircraft  Sepa¬ 
ration,  Landing  and  Departure  Operations. 


1  Introduction 

Small  Aircraft  Transportation  System  (SATS)  [IB],  developed  by  NASA,  pro¬ 
vides  access  to  more  communities  with  less  time  delays  by  leveraging  upon  the 
recent  advances  in  navigation  and  communication  technologies.  When  a  number 
of  aircraft  are  in  different  parts  of  the  airport,  aircraft  safety  has  to  be  ensured 
through  timely  separation  and  sequencing.  Traditionally,  non-towered  non-radar 
airports  rely  on  procedural  separation  during  Instrument  Meteorological  Condi¬ 
tions  (IMC),  i.e.,  allowing  only  one  aircraft  to  get  access  to  the  airport  airspace 
at  a  given  time,  which  significantly  decreases  the  potential  airport  throughput 
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[23].  The  main  objective  of  SATS  is  to  facilitate  high  volume  operations  (HVO) 
of  advanced  small  aircraft  at  such  airports  with  minimum  infrastructure  and  low 
cost.  Some  representative  SATS  aircraft  are  Very  Light  Jet  (VLJ)  aircraft,  an  ad¬ 
vanced  technology  Single-Engine  (SE),  piston-powered  aircraft  and  an  advanced 
technology  Multi-Engine  (ME),  piston-powered  aircraft  [33i . 

Conventionally,  SATS  HVO  simulations  have  been  performed  using  computer 
programs  in  which  aircraft  modules  were  operated  manually  by  pilots.  These 
simulations  develop  the  human-in-tlie-loop  scenarios  to  check  the  effect  of  SATS 
procedures  in  the  operational  environment,  on  the  pilot’s  responses  in  terms  of 
work  load  and  situational  awareness  [31112116132) .  In  [T2] ,  off-nominal  situations 
were  also  simulated,  in  addition  to  the  nominal  situations,  to  check  the  result¬ 
ing  effect  on  the  pilot’s  state  of  mind.  Proof-of-concept  simulation  studies  were 
performed  in  the  Air  Traffic  Control  (ATC)  simulation  pilot  lab  at  Federal  Avi¬ 
ation  Administration  William  J.  Hughes  Technical  Center  (FAATC)  [30] .  These 
simulations  validated  that  the  ATC  can  accept  the  SATS  procedures,  are  able 
to  control  SATS  traffic  into  and  out  of  the  Self  Controlled  Area  (SCA),  and 
support  high  volume  operations.  The  simulations  with  pilots  were  used  only  for 
validation  purposes  and  confirmed  that  SATS  procedures  are  manageable  by  the 
airport  management  module  (AMM).  AMM’s  performance  during  high  arrival 
rates  of  aircraft  into  the  SCA  has  also  been  studied  and  found  to  have  less  delays 
as  compared  to  one-in-one-out  method  E3-  Recently,  an  algorithm  has  been  de¬ 
veloped  to  optimize  SATS  landing  sequence  for  multiple  aircraft  in  [3],  to  make 
it  conflict-free  and  with  less  delays,  using  Microsoft  VC++  6.0  simulation  en¬ 
vironment.  However,  these  piloted  simulation  methods  lack  exhaustiveness  m 
in  terms  of  coverage  of  all  the  possible  states  as  a  rigorous  piloted  simulation  of 
all  possible  scenarios  requires  a  large  number  of  tests,  which  in  turn  demands 
a  significant  amount  of  computational  power  and  time.  This  leads  to  another 
major  challenge  of  simulation-based  verification  of  the  SATS  Concept  of  Opera¬ 
tions  (ConOps),  i.e. ,  selection  of  test  vectors.  A  random  selection  of  test  vectors 
cannot  offer  a  guarantee  of  correctness  of  the  SATS  ConOps  since  it  might  miss 
the  meaningful  portion  of  the  design  space.  Moreover,  it  may  not  be  possible  to 
consider  or  even  foresee  all  corner  cases.  Consequently,  simulation-based  verifi¬ 
cation  of  the  SATS  ConOps  is  incomplete  with  respect  to  error  detection,  i.e., 
all  errors  in  a  system  cannot  be  guaranteed  to  be  detected,  which  is  a  severe 
limitation  considering  the  safety-critical  nature  of  passenger  aircraft. 

In  order  to  have  a  complete  analysis,  automatic  parameterized  verification  of 
hybrid  automata  [20119]  was  recently  employed  to  verify  properties  of  the  SATS 
ConOps  using  model  checking  principles,  while  considering  position  of  the  air¬ 
craft  as  a  continuous  variable  modeled  either  as  a  timer  m  or  as  a  rectangular 
differential  inclusion  [20] .  While  this  methodology  allows  for  verification  regard¬ 
less  of  the  number  of  aircraft,  a  limitation  of  this  work  is  that  the  methodology 
requires  the  user  to  specify  inductive  invariants  sufficient  to  establish  safety. 
While  the  process  of  finding  inductive  invariants  sufficient  to  establish  safety 
of  the  SATS  ConOps  has  been  successfully  automated  through  an  extension  of 
invisible  invariants  0,  this  is  an  incomplete  (heuristic)  method  that,  in  general, 
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may  fail  to  find  such  inductive  invariants  [21|.  The  analysis  and  formal  verifica¬ 
tion  of  the  timing  constraints  of  SATS  was  done  in  JTOj  using  Linear  Real-Time 
Logic  (LRTL).  The  higher-order-logic  theorem  prover  PVS  J2B]  has  also  been 
used  for  the  safety  verification  of  the  SATS  ConOps  [13I9I23I2T)].  In  particu¬ 
lar,  it  has  been  formally  verified  that  SATS  rules  and  procedures  can  provide 
minimum  required  spacing  between  two  and  more  aircraft.  A  hybrid  modeling 
technique  was  also  developed  in  PVS  using  the  PVS  tool  Besc  [25] . 

In  the  above-mentioned  methods  of  validation  and  verification  of  SATS,  only 
the  procedures  and  transition  rules  are  considered.  With  these  considerations, 
any  model  with  appropriate  conditions  can  validate  that  the  procedures  are 
enough  for  the  assurance  of  safe  separation  between  the  aircraft.  The  missed 
approach  transition  is  dependent  on  many  random  factors,  for  instance,  low  vis¬ 
ibility.  In  conventional  airports,  it  is  mainly  caused  by  the  bad  weather,  increased 
air-borne  traffic  density,  and  ground  traffic  and  its  delays  m-  It  is  also  required 
upon  the  execution  of  a  rejected  landing  because  of  objects,  such  as  men,  equip¬ 
ment  or  animals,  on  the  runway  Ij.  Due  to  such  uncertainties  involved,  it  is 
necessary  to  incorporate  the  probabilistic  considerations  of  the  system  into  the 
validation  methods  and  safety  verifications  of  SATS.  Hence,  we  propose  to  use 
probabilistic  model  checking  for  the  verification  of  the  SATS  ConOps.  This 
paper  presents  a  fully  synchronous  Discrete-Time  Markov  Chain  (DTMC)  model 
of  the  SATS  ConOps  and  the  verification  of  the  safety  properties  of  SATS,  includ¬ 
ing  the  landing  and  take-off  procedures,  using  the  probabilistic  model  checker 
PRISM  [22].  PRISM  has  been  extensively  used  to  formally  model  and  analyze 
a  wide  variety  of  systems,  including  communication  and  multimedia  protocols, 
randomised  distributed  algorithms,  security  protocols,  biological  systems  and 
many  others,  that  exhibit  random  or  probabilistic  behaviour  [2. 

The  rest  of  the  paper  is  organized  as  follows:  Section  [2]  describes  the  SATS 
operational  concept  to  facilitate  the  understanding  of  the  rest  of  the  paper.  Sec¬ 
tion  [3]  explains  the  main  challenges  that  we  faced  in  modeling  the  considered, 
fully  synchronous,  system  in  PRISM  and  the  assumptions  used  in  our  DTMC 
model.  In  this  section,  our  modeling  methodology  is  also  explained  through  dis¬ 
cussion  about  each  module,  transition  rules  and  procedures.  Section  [4]  presents 
the  probabilistic  verification  results  of  the  SATS  ConOps  and  the  novel  observa¬ 
tions  made.  Finally,  Section  [5]  concludes  this  paper  by  drawing  conclusions  and 
mentioning  some  directions  of  future  work. 


2  SATS  ConOps 

The  ConOps  for  SATS  is  primarily  a  set  of  rules  and  procedures  based  on  an 
area  surrounding  the  airport,  called  the  SC  A,  a  centralized  automated  system, 
called  the  AMM,  data  communication  between  AMM  and  aircraft  and  state  data 
broadcast  from  the  aircraft  03-  The  SCA  is  typically  taken  as  a  region  with  12- 
15  nautical  miles  radius  and  3000  feet  above  the  ground  18101 .  It  is  arranged  in  a 
T  structure,  consisting  of  base,  intermediate  and  final  zones.  It  is  divided  into  a 
number  of  segments  and  fixes  which  are  the  latitude/ longitude  points  in  space. 
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IF  IAF-L 


Fig.  1:  Top  view  of  the  SCA  |13j 


The  fixes  are  initial  arrival  fixes  (IAFs),  intermediate  fix  (IF),  final  approach 
fix  (FAF)  and  departure  fixes  (DFs),  as  shown  in  Fig.  [lj  The  IAFs  serve  two 
purposes,  i.e. ,  holding  fix,  when  an  aircraft  enters  the  SCA,  and  missed  approach 
holding  fix  (MAHF),  which  is  required  when  an  aircraft  misses  landing,  and  flies 
back  to  the  IAF  via  missed  approach  path. 

There  are  two  types  of  entries  into  the  SCA:  vertical  entry  and  lateral  entry 
EES],  as  depicted  in  Fig.  [2]  Vertical  entry  is  always  made  from  the  3000  feet 
holding  fix  at  the  left  (above  IAF-L)  or  right  (above  IAF-R).  Thereafter,  the  air¬ 
craft  descends  to  the  respective  2000  feet  holding  fix  when  it  becomes  available. 
Next,  under  certain  conditions,  the  aircraft  moves  to  the  base  segment  (IAF  to 
IF).  On  the  other  hand,  in  a  lateral  entry,  the  aircraft  flies  from  the  point  of 
entry  to  the  base  segment  directly  or  through  the  2000  feet  holding  fix.  Once 
the  aircraft  is  in  the  base  segment  or  2000  feet  holding  fix,  there  is  no  depen¬ 
dency  on  its  type  of  entry.  After  base  segment,  the  aircraft  goes  through  the  IF, 
FAF,  and  finally  reaches  the  runway.  This  procedure  is  primarily  composed  of  a 
series  of  transitions  through  different  segments  of  the  SCA  that  are  conducted 
by  the  aircraft  if  sufficient  separation  from  the  other  aircraft  is  available  and  all 
conditions  for  the  given  transitions  hold.  If  an  aircraft  misses  its  landing,  due 
to  any  reason,  it  has  to  follow  the  missed  approach  path  to  move  to  the  IAF 
corresponding  to  its  MAHF  assignment,  as  shown  in  Fig.  [I] 

The  AMM  has  the  responsibility  to  grant  permissions  to  the  aircraft  for 
entering  the  SCA  dai].  While  granting  the  permission,  the  AMM  assigns  a 
landing  sequence  and  a  MAHF  to  the  aircraft.  These  landing  sequence  numbers 
encode  the  leader  information  and  also  identify  whether  an  aircraft  is  the  first 
aircraft  in  a  specific  zone  of  SCA.  The  aircraft  entering  later  thus  follows  the 
leader  during  the  transitions.  The  MAHF  assignment  is  in  terms  of  ‘side’,  which 
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Lateral  Entry 


Vertical  Entry 


5000  feet 
4000  feet 
3000  feet 
2000  feet 


IAF-L 


Runway  I 

FAF  IAF-R 

IF 


Fig.  2:  Side  view  of  the  SCA  [13] 


h3-R  <-  ->  g-  h3-L 


h2-R  ==>■==  h2-L 


Fig.  3:  Zones  of  the  SCA  13] 


can  assume  values  of  right  or  left.  If  the  entering  aircraft  is  the  first  one  in 
sequence,  then  its  MAHF  will  be  in  the  same  side  from  which  it  is  entering. 
Whereas,  the  next  aircraft,  with  sequence  other  than  1,  will  have  the  MAHF 
that  is  opposite  to  that  of  its  leader. 

Departure  fixes  are  outside  the  SCA  and  under  the  ATC  control.  An  air¬ 
craft  ready  to  depart  requests  ATC  for  clearance.  After  clearance,  the  departure 
operation  starts  at  the  runway  and  it  moves  to  the  departure  fix  correspond¬ 
ing  to  its  MAHF  assignment.  A  safe  distance  of  10  or  5  nautical  miles  has  to 
be  maintained  from  the  aircraft  flying  to  the  same  or  opposite  departure  fixes, 
respectively  m 

The  SCA  can  be  divided  into  different  zones,  illustrated  in  Fig.  [3]  and  pre¬ 
sented  in  Table  [I]  These  zones  represent  the  state  of  the  aircraft.  The  complete 
information  about  the  aircraft  will  thus  include  the  sequence  and  MAHF  as¬ 
signed  by  AMM  and  the  current  location/zone  of  aircraft.  The  safety  verifica¬ 
tion  is  based  on  the  number  of  aircraft  in  a  zone  and  their  separation  from  other 
aircraft  in  other  zones  [23], 
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Table  1:  Zones  of  SCA  [13] 


Zone 

Symbol 

Description 

1 

h3-R 

Holding  at  3000  feet  at  right  side 

2 

h3-L 

Holding  at  3000  feet  at  left  side 

3 

h2-R 

Holding  at  2000  feet  at  right  side 

4 

h2-L 

Holding  at  2000  feet  at  left  side 

5 

lez-R 

Lateral  entry  zone  at  right  side 

6 

lez-L 

Lateral  entry  zone  at  left  side 

7 

base-R 

Right  segment  of  base  (IAF-R  to  IF) 

8 

base-L 

Left  segment  of  base  (IAF-L  to  IF) 

9 

int 

Intermediate  segment  (IF  to  FAF) 

10 

fin 

Final  segment  (FAF  to  runway) 

11 

run 

Runway 

12 

maz-R 

Missed  approach  zone  at  right  of  base 

13 

maz-L 

Missed  approach  zone  at  left  of  base 

14 

taxi 

Taxi 

15 

dep-R 

Right  departure  path  towards  right  departure  fix 

16 

dep-L 

Departure  path  towards  left  departure  fix 

3  Formal  Modeling  of  SATS  as  a  DTMC  in  PRISM 

In  this  section,  we  first  describe  our  refinements  to  the  SATS  ConOps.  Then  the 
main  challenges  encountered  in  modeling  the  system  in  PRISM  are  presented. 
This  is  followed  by  the  description  of  how  these  challenges  were  tackled  in  our 
model. 


3.1  Refinements  to  original  SATS 

The  proposed  model  of  the  SATS  ConOps  in  the  PRISM  language  overcomes 
some  of  the  limitations  of  the  non-deterministic,  asynchronous  transition  system 
presented  by  Dowek  et.  al  [T5].  Before  presenting  the  details  of  our  model,  we 
find  it  appropriate  to  point  out  the  discrepancies  in  the  existing  algorithm  and 
our  proposed  solution. 

1.  In  a  non-deterministic  model,  if  two  or  more  rules  are  enabled  simultaneously, 
any  one  of  them  is  allowed  to  be  executed.  In  other  words,  only  one  non- 
deterministic  action  happens  at  a  time.  This  means  that  in  such  a  model, 
at  each  time  step,  only  one  aircraft  will  move  to  the  next  zone  while  all 
other  aircraft  hold  in  the  same  zone,  even  if  the  conditions  are  satisfied  for 
all  aircraft  to  move  to  their  respective  next  zones.  Thus,  one  aircraft  could 
change  zones  several  times  while  another  remains  idle  1 1 31 .  Hence,  such  a 
model  is  unrealistic  [53],  as  it  fails  to  depict  the  real  scenario. 

2.  The  lowest  available  altitude  determination  (Rule  12)  [13]  is  a  simultane¬ 
ous  transition,  potentially  involving  2  aircraft,  when  the  holding  pattern  at 
3000  feet  is  occupied  but  2000  feet  is  available.  In  this  case,  the  transition 
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determines  3000  feet  as  the  lowest  available  altitude  and  forces  the  aircraft 
holding  at  3000  feet  to  descend  to  the  holding  pattern  at  2000  feet.  This  is 
a  weakness  of  the  model  because  simultaneous  transition  is  not  possible  in 
a  fully  non-deterministic  model. 

Our  proposed  solution  for  both  the  above  limitations  is  to  build  a  fully  syn¬ 
chronous  model  that  allows  simultaneously  moving  aircraft.  Hence,  at  each  time 
step,  all  aircraft  satisfying  conditions  to  move  to  their  respective  next  zones 
are  allowed  to  proceed  concurrently.  Moreover,  this  model  also  facilitates  the 
simultaneous  transition  in  the  lowest  available  altitude  determination. 


3.2  Modelling  Challenges  of  SATS  in  PRISM 


Parallel  Composition  of  Modules 

Parallel  composition  of  modules  in  PRISM  may  seem  to  be  the  best  option  for 
developing  the  interleaved  model  of  concurrency  of  aircraft  in  the  SCA,  where 
each  module  represents  an  aircraft.  However,  there  are  critical  limitations  in 


such  a  model,  as  discussed  in  Section  3.1  When  multiple  commands  (belonging 


to  any  of  the  modules)  are  enabled  at  the  same  time,  the  choice  between  which 
command  is  executed  by  PRISM  is  non-deterministic  in  case  of  Markov  decision 
process  (MDP)  and  probabilistic  in  case  of  DTMC  [3j.  Specifically  in  the  case 
of  a  DTMC,  PRISM  selects  the  command  for  execution  uniformly  at  random. 
For  instance,  if  there  are  4  aircraft  in  the  SCA  and  guards  are  satisfied  for  one 
command  in  each  module,  then  there  is  a  probability  of  0.25  for  each  aircraft  to 
move  forward  to  the  next  zone.  But  only  one  of  them  is  selected  to  move  at  a 
time. 


Synchronization 

PRISM  supports  synchronized  transitions  using  synchronization  labels.  In  this 
case,  commands  can  be  labelled  with  actions,  which  can  be  used  to  force  two 
or  more  modules  to  make  transitions  simultaneously.  By  default,  all  modules 
are  combined  using  the  standard  CSP  parallel  composition,  i.e.,  modules  syn¬ 
chronize  over  all  their  common  actions  [2|.  However,  in  SATS  application,  the 
aircraft  can  be  in  any  of  the  16  zones  and  thus  only  a  specific  scenario  can  be 
modelled  using  synchronization  labels.  For  instance,  if  there  are  two  aircraft  and 
the  command  for  the  first  aircraft  to  be  in  the  third  zone  is  synchronized  with 
the  command  for  the  second  aircraft  to  be  in  the  first  zone,  then  they  will  make 
the  transition  simultaneously,  if  available,  but  it  models  a  special  case  out  of 
the  many  possibilities.  They  will  no  longer  be  synchronized  in  some  future  time 
step  when  the  first  aircraft  is,  for  instance,  in  the  seventh  zone  while  the  second 
aircraft  is  in  the  first  zone. 


Global  variables  with  Synchronization 

Global  variables  seem  useful  in  modelling  the  state  of  the  aircraft  in  the  SCA 
as,  unlike  local  variables,  they  are  modifiable  from  any  module.  However,  an 
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important  restriction  on  the  use  of  global  variables  in  PRISM  is  the  fact  that 
global  variables  cannot  be  updated  on  a  synchronized  command  [2j.  PRISM 
detects  this  and  reports  an  error  if  an  attempt  is  made  to  do  so. 


Probabilistic  Updates 

In  order  to  correctly  model  the  semantics  of  the  communication  between  aircraft 
and  AMM,  both  aircraft  and  AMM  should  have  separate  modules  in  PRISM. 
Unfortunately,  there  is  no  direct  way  of  changing  a  variable  in  a  different  module 
for  only  one  probabilistic  update  of  a  command  in  the  same  time  step.  However, 
such  probabilistic  updates  are  frequently  required.  For  instance,  when  an  aircraft 
is  in  the  final  zone  and  it  can  move  to  the  runway  or  missed  approach  path  with 
certain  probabilities.  In  case  a  pilot  chooses  the  missed  approach  path,  a  new 
sequence  number  is  to  be  assigned  to  the  aircraft  by  the  AMM  while  in  case 
of  transition  to  runway,  there  is  no  change  in  the  sequence  number.  A  possible 
solution  could  be  to  change  the  model  such  that  the  relevant  variable  is  part  of 
the  same  module  as  the  probabilistic  update  but  it  will  not  represent  the  actual 
scenario  of  the  communication  between  aircraft  and  the  AMM. 

Therefore,  the  challenge  is  to  achieve  a  synchronization  such  that  all  aircraft 
move  together  whenever  the  guard  conditions  are  satisfied,  while  incorporating 
probabilistic  updates  from  the  AMM  in  the  model. 


3.3  Modeling  SATS  in  PRISM 

In  our  formal  model  [28],  we  formulate  the  SATS  ConOps  as  a  DTMC  in  the 
PRISM  model  checker  using  an  abstract  timing  model.  Both  sides  of  the  ap¬ 
proach  are  symmetric  H5I25I  and  there  can  be  at  most  two  aircraft  on  each  side 
of  the  SCA  [13123] .  Therefore,  we  have  assumed  two  aircraft  in  the  right  side  of 
the  SCA  in  this  work  for  the  purpose  of  simplicity.  Our  model  ensures  that  after 
a  landing  aircraft  has  landed  safely,  it  unloads  passengers  of  the  current  flight 
in  the  taxi  state.  Then,  it  loads  passengers  of  the  next  flight  and  is  ready  for  de¬ 
parture.  After  departure,  it  reaches  its  destination  and  the  next  time  it  becomes 
a  landing  aircraft  for  the  SCA.  Hence,  the  process  of  landing  and  departure 
continues. 


Model  of  Concurrency 

In  order  to  cope  with  the  challenges,  described  in  Section  [372]  we  modeled  the 
SATS  ConOps  as  fully  synchronously  parallel  automata,  as  in  m,  where  each 
transition  is  labeled  with  the  same  synchronization  label,  and  therefore  at  each 
time  step,  at  least  one  transition  of  each  module  is  active.  Hence,  in  such  a  fully 
synchronous  model,  both  aircraft  move  concurrently  to  the  next  respective  zones 
whenever  the  conditions  are  satisfied.  In  order  to  use  the  same  synchronization 
label  t  with  all  commands  in  all  modules,  we  ensure  that  at  least  one  condition 
is  true  for  each  module  for  each  reachable  state  in  our  model. 
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Model  of  SATS  Transition  Rules  and  Procedures 

The  modules  aircraft  1  and  aircraft2  in  our  formal  model  }28j.  corresponding 
to  each  aircraft,  implement  the  rules  of  ConOps,  i.e. ,  under  what  conditions  the 
aircraft  moves  from  one  zone  to  the  next.  The  modules  are  symmetric  except 
that  priority  is  assigned  to  aircraft  1  in  case  of  simultaneous  entry.  Due  to  our 
proposed  fully  synchronous  model,  aircraft  can  enter  inside  the  SCA  individually 
or  simultaneously  with  another  aircraft.  The  state  variables  zonel  and  zone2 
represent  the  current  zone  of  aircraft  1  and  aircraft2,  respectively.  They  are 
modelled  as  integer  variables  with  values  in  the  range  0  -  16,  and  the  encoding  is 
listed  in  Table  [1]  One  additional  zone  is  to  be  included  into  the  model,  which  is 
the  ‘fly  zone’,  for  an  aircraft  outside  the  SCA.  We  encode  it  with  a  value  of  zero. 
In  our  model,  we  used  formulas  for  compact  representation  of  the  conditions 
and  to  avoid  repetition.  For  instance,  zl_total  represents  the  total  number  of 
aircraft  in  zone  1  and  z7_total_R  represents  number  of  aircraft  in  zone  7  with 
an  MAHF  assignment  of  right,  as  shown  in  the  following  lines  of  the  code  in 
PRISM  language: 

formula  zlUotal  =  (zone  1  =  1?1  :  0)  +  (zone 2  =  1?1  :  0); 

formula  zl-total-R  =  (zone  1  =  7  &  mahf  1  =  truel  1  :  0) 

+  (zone 2  =  7  &  mahf2  =  truel  1  :  0); 


Model  of  the  AMM 

The  AMM  is  the  sequencer  of  the  SCA.  It  typically  resides  at  airport  ground  and 
communicates  with  the  aircraft  via  a  data  link  [8] .  We  model  AMM  as  a  separate 
module  AMM  in  PRISM  to  represent  this  communication  with  the  aircraft.  It  has 
two  state  variables,  i.e.,  seq  and  mahf  for  each  aircraft.  For  a  landing  aircraft, 
seq  represents  the  relative  landing  sequence  number,  such  that  the  aircraft  with 
landing  sequence  n  is  the  leader  of  the  aircraft  with  landing  sequence  n+1,  i.e., 
an  aircraft  with  sequence  number  1  is  leader  of  the  aircraft  with  sequence  number 
2.  It  is  modelled  as  an  integer  variable  with  values  in  the  range  0  -  10.  When  an 
aircraft  enters  the  SCA,  seq  is  assigned  a  new  value  calculated  by  the  formula 
next  seq.  This  value  is  calculated  based  on  the  number  of  the  aircraft  already 
in  the  landing  zones  of  the  SCA.  In  case  of  simultaneous  entry  by  both  aircraft, 
different  sequence  numbers  are  assigned  to  both  the  aircraft,  with  priority  to 
aircraftl.  A  new  sequence  number  is  also  assigned  when  an  aircraft  initiates 
a  missed  approach  path  and  the  sequence  numbers  of  all  other  aircraft  in  the 
landing  zones  of  the  SCA  are  decremented  by  one.  Moreover,  when  an  aircraft 
enters  runway,  the  sequence  numbers  of  all  other  aircraft  in  the  SCA  are  again 
decremented  by  one.  When  an  aircraft  moves  to  the  taxi  state,  its  sequence 
number  becomes  0.  For  a  departing  aircraft,  seq  represents  the  distance  of  the 
aircraft  from  runway  in  nautical  miles.  It  is  incremented  by  one  in  each  time 
step  when  it  is  in  one  of  the  departure  zones,  until  it  becomes  10,  where  it  is 
assumed  to  have  left  the  SCA.  The  MAHF  of  an  aircraft,  represented  by  mahf,  is 
a  boolean  variable  with  true  representing  right  MAHF,  and  false  representing 
left  MAHF.  It  is  assigned  whenever  an  aircraft  enters  the  SCA.  Moreover,  it 
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is  re-assigned  when  an  aircraft  executes  a  missed  path  approach.  We  consider 
MAHF  of  only  right  side  for  simplicity  of  the  model  in  this  paper. 


Timing  Model 

We  use  an  abstract  timing  model  in  our  formalization  of  the  SATS  ConOps. 
We  assume  that  each  aircraft  stays  in  a  zone  for  at  least  one  time  step.  So,  an 
aircraft  must  transition  to  the  next  zone  after  one  time  unit  if  the  conditions  for 
transition  are  satisfied.  When  the  guard  conditions  are  not  fulfilled,  it  stays  in 
the  zone  until  the  conditions  become  true. 


Randomness  in  Model 

Since  there  is  no  direct  way  of  changing  a  variable  in  a  different  module  for  only 
one  probabilistic  update  of  a  command  in  the  same  time  step,  we  introduce  an 
additional  chooser  module  for  each  probabilistic  decision.  For  instance,  consider 
an  aircraft  in  the  final  zone.  Now  it  can  either  choose  the  missed  approach  path 
with  a  probability  p_map  or  it  can  continue  landing  and  transit  to  the  runway 
with  probability  l-p_map.  In  case  of  the  missed  approach  path,  a  new  sequence 
number  and  MAHF  is  to  be  assigned  to  the  aircraft.  However,  there  is  no  change 
in  its  sequence  number  and  MAHF  if  it  proceeds  to  runway.  We  propose  to 
use  the  chooser  module,  choose_p_map  which  contains  a  single  state  variable 
p_map_state  of  type  integer  and  with  two  possible  values:  0  and  1.  When  the 
probability  p_map  is  selected,  p_map_state  is  set  to  1,  otherwise  it  is  0.  This  is 
achieved  by  using  the  following  command  in  PRISM: 

[t]  Guard  — >  p-map  :  ( p-mapstate  =  1)  +  (1  —  p-map )  :  ( pjmapstate  =  0); 

It  is  important  to  note  that  instead  of  setting  true  as  a  guard,  we  use  the 
conditions  of  transition  to  final  zone,  i.e.,  one  step  back  condition  as  the  guard 
|28] .  This  way,  the  command  does  not  execute  on  each  time  step.  p_map_state 
is  updated  when  the  aircraft  enters  the  final  zone  and  is  ready  to  be  used  when 
checking  conditions  for  the  next  transition  to  runway  or  missed  approach  zone 
in  the  next  time  step. 

The  value  of  p_map_state  is  now  used  in  such  a  way  that  the  guard  condition 
of  p_map_state=l  checks  whether  p_map  is  selected.  For  instance,  in  the  AMM 
module,  the  following  command  ensures  that  seql  and  mahfl  are  updated  as 
soon  as  it  makes  the  transition  to  zone  12: 

[t]  Guard  &  pjmapstate  =  1  — >  (seql1  =  nextseq)  &  (mahfl1  =  nextmahfl ); 

4  Verification  Results 

4.1  Safety  Properties 

Based  on  our  model,  explained  in  Section  [3j  safe  separation  is  not  maintained 
when  two  aircraft  reside  simultaneously  in  the  specific  zones.  These  zones  include 
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the  approach,  final  approach,  missed  approach,  runway  and  departure  zones. 
Hence,  we  label  this  state  danger  as  follows: 

label  “ danger ”  =  ((zone  1  =  7$zzone2  =  7)  |  (zonel  =  9&zone2  =  9) 

|  (zone  1  =  10&;zone2  =  10)  |  (zone  1  =  ll&;zone2  =  11) 

|  (zone  1  =  Y2fzzone2  =  12)  |  (zone  1  =  15&;zone2  =  15)); 

Safety  in  all  Paths:  P  =?  [F  “danger”]; 

We  analyze  safety  in  our  model  using  the  above  property,  which  computes 
the  value  of  the  probability  that  danger  is  satisfied  in  the  future  by  the 
paths  from  the  initial  state.  PRISM  shows  a  result  of  0,  which  confirms  that 
no  path  leads  to  a  collision  from  the  initial  state. 

Safety  in  all  Reachable  States:  filter  (forall,P  <=  0  [F  “danger”]); 

In  order  to  confirm  that  the  probability  of  occurrence  of  danger  remains 
0  for  all  reachable  states,  we  formalize  the  property  using  filters  as  above. 
The  property  verifies  to  be  true  in  PRISM  and  thus  guarantees  the  safety  in 
our  model. 

4.2  Analysis  of  Landing  and  Departure  Operations 
Expected  Time  for  Landing:  R  =?  [F  “landing si”]; 

We  utilize  the  reachability  reward  [2]  in  PRISM  to  find  the  expected  time  taken 
for  the  landing  of  an  aircraft  in  our  model.  In  this  case,  a  reward  of  unity  is 
awarded  to  each  state  of  the  model  and  the  rewards  are  accumulated  along  a 
path  until  a  certain  point  is  reached.  We  define  this  point  as  the  state  in  which 
the  aircraft  is  in  the  taxi  state,  for  instance,  for  aircraft  1: 

label  “ landingsl ”  =  (zone  1  =  14); 

Since  very  limited  information  is  available  on  the  probability  of  executing  a 
missed  approach  path  p_map  for  SATS,  we  leverage  upon  the  PRISM’s  parametric 
model  checking  functionality  to  perform  the  sensitivity  analysis  on  the  values 
of  punap  from  0.001  to  0.9.  The  results  are  shown  in  Fig.  [4j  which  depict  the 
exponential  increase  in  the  expected  time  taken  for  landing  with  p_map.  Since 
aircraft  1  is  assigned  priority  in  case  of  simultaneous  entry,  the  values  for  this 
aircraft  are  slightly  smaller  as  compared  to  those  of  aircraft2.  The  overall 
expected  time  for  any  aircraft  to  land  is  also  shown. 

Expected  Number  of  Departures  in  a  Fixed  Time:  R  =1  [C  <=  T]; 

We  leverage  upon  the  cumulative  reward  properties  [2]  to  find  the  expected 
number  of  departures  of  the  aircraft  in  a  fixed  time  in  our  model.  In  this  case, 
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Probability  of  Missed  Approach  Zone  (p_map) 


—  aircraftl 

—  aircraft2 

—  overall 


Fig.  4:  Expected  time  for  landing  vs.  Probability  of  the  Missed  Approach  Zone 


a  reward  of  unity  is  awarded  to  each  transition  of  departure  and  the  rewards 
are  accumulated  until  T  time  steps  have  elapsed.  Fig.  [5]  shows  the  results  of  an 
experiment  with  T  set  to  10,00,000  which  is  large  enough  for  the  purpose  of  com¬ 
parative  analysis.  Since  aircraftl  is  assigned  priority  in  case  of  simultaneous 
departure,  the  expected  number  of  departures  for  this  aircraft  are  slightly  larger 
as  compared  to  those  of  aircraft2. 

Comparison  of  SATS  and  SATS-SMA:  Reproduction  of  the  correspond¬ 
ing  non-deterministic  model  [13]  in  PRISM  shows  that  the  expected  number  of 
landing  or  departure  operations  are  much  greater  in  our  proposed  SATS-SMA 
than  the  corresponding  non-deterministic  model.  For  instance,  with  no  aircraft 
executing  a  missed  approach  path,  i.e. ,  punap  of  0,  the  expected  operations  in 
the  original  non-deterministic  asynchronous  model  and  our  refined  SATS-SMA 
are  51280  and  81081,  respectively,  i.e.,  around  1.6  times  greater  throughput. 
The  reason  is  that  original  SATS  allows  only  one  aircraft  to  move  at  a  time 
while  we  allow  all  aircraft  satisfying  the  conditions  to  move  simultaneously  to 
the  respective  next  zones. 

The  key  advantages  of  this  work  include  the  increase  in  the  throughput, 
while  maintaining  aircraft  safety,  through  simultaneous  operations.  The  work 
also  provides  important  quantitative  landing  and  departure  insights  of  the  SATS 
ConOps.  Our  PRISM  code  and  properties  file  is  available  for  download  [25], 
and  thus  can  be  benefited  by  researchers  and  verification  engineers  for  further 
developments  and  analysis  of  the  SATS  ConOps. 

5  Conclusion 

Given  the  random  and  unpredictable  nature  of  entry  of  aircraft  into  the  SCA  and 
transitions  between  the  zones,  we  propose  to  use  a  probabilistic  model  checker, 
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Fig.  5:  Expected  departures  vs.  Probability  of  the  Missed  Approach  Transition 


PRISM,  to  analyze  the  SATS  ConOps  in  this  paper.  A  fully  synchronous  DTMC 
model  of  SATS  is  proposed  and  is  verified  to  increase  the  expected  throughput 
of  the  airport  as  compared  to  the  traditional  non-deterministic,  asynchronous 
model.  Moreover,  the  successful  modeling  and  verification  of  the  transition  pro¬ 
cedures  for  two  aircraft  moving  concurrently,  has  verified  the  safety  of  aircraft  in 
terms  of  safe  separation  in  all  zones  including  take-off  and  landing.  The  landing 
and  departure  operations  of  SATS  are  analyzed  with  respect  to  the  probability 
associated  with  the  missed  approach  transition. 

An  important  direction  of  future  work  is  to  improve  the  timing  model  by 
incorporating  zone  distances  and  abstract  aircraft  kinematics  [25].  A  more  de¬ 
tailed  analysis  can  be  carried  out  by  removing  the  simplifying  assumptions  of 
2  aircraft  and  right  side  MAHF.  Similarly,  detailed  comparison  of  non-SATS 
(one- in/one-out),  SATS  and  SATS-SMA  is  an  interesting  direction  for  future 
research.  Furthermore,  we  also  plan  to  conduct  the  probabilistic  analysis  of  the 
SATS  ConOps  under  off-nominal  conditions  |24lfil12j.  such  as  equipment  mal¬ 
function  and  emergency  situations,  using  the  parametric  model  checking  func¬ 
tionality  of  PRISM,  like  it  was  utilized  for  the  analysis  of  probability  of  missed 
approach  in  this  paper.  Moreover,  Continuous-Time  Markov  Chains  (CTMCs)  of 
the  SATS  ConOps  can  also  be  developed  to  verify  some  time-related  properties, 
where  Erlang  distribution  can  be  used  to  model  discrete  time  delays  El- 
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Abstract 

Safety  verification  of  hybrid  dynamical  systems  relies  crucially  on  the  ability  to  reason  about  reach¬ 
able  sets  of  continuous  systems  whose  evolution  is  governed  by  a  system  of  ordinary  differential  equa¬ 
tions  (ODEs).  Verification  tools  are  often  restricted  to  handling  a  particular  class  of  continuous  systems, 
such  as  e.g.  differential  equations  with  constant  right-hand  sides,  or  systems  of  affine  ODEs.  More  re¬ 
cently,  verification  tools  capable  of  working  with  non-linear  differential  equations  have  been  developed. 
The  behavior  of  non-linear  systems  is  known  to  be  in  general  extremely  difficult  to  analyze  because 
solutions  are  rarely  available  in  closed-form.  In  order  to  assess  the  practical  utility  of  the  various  veri¬ 
fication  tools  working  with  non-linear  ODEs  it  is  very  useful  to  maintain  a  set  of  verification  problems. 
Similar  efforts  have  been  successful  in  other  communities,  such  as  automated  theorem  proving,  SAT 
solving  and  numerical  analysis,  and  have  accelerated  improvements  in  the  tools  and  their  underlying 
algorithms.  We  present  a  set  of  65  safety  verification  problems  featuring  non-linear  polynomial  ODEs 
and  for  which  we  have  proofs  of  safety.  We  discuss  the  various  issues  associated  with  benchmarking 
the  currently  available  verification  tools  using  these  problems. 


1  Introduction 

For  verifying  safety  properties  of  hybrid  systems,  it  is  crucial  to  have  the  means  of  reasoning 
about  safety  properties  of  purely  continuous  systems  that  determine  state  evolution  inside  the 
operating  modes. 

In  computer  science,  emphasis  has  traditionally  been  placed  on  working  with  hybrid  systems 
in  which  the  continuous  modes  are  governed  by  relatively  simple  ODEs.  For  instance,  safety 
verification  of  systems  with  ODEs  possessing  constant  right-hand  sides  and  right-hand  sides 
bounded  within  real  intervals  is  aided  by  the  fact  that  reachable  sets  of  such  continuous  systems 
can  be  computed  exactly.  Progress  has  been  made  on  verifying  safety  in  systems  with  linear 
and  affine  continuous  dynamics  (with  tools  such  as  PHAVer  [13]  and  SpaceEx  [14]).  This 
is  a  much  more  difficult  problem,  since  reachable  sets  of  linear  ODEs  cannot  in  general  be 
phrased  in  a  decidable  theory,  which  is  only  known  to  be  possible  for  some  special  classes  of 
systems  [22,  16,  18]. 

It  is  a  well-known  fact  that  non-linear  ODEs  can  exhibit  behaviour  that  is  impossible  under 
affine  or  linear  dynamics  [19].  Their  expressive  power  allows  for  modelling  very  rich  dynamic 
phenomena,  but  comes  at  the  price  of  making  the  reachability  analysis  much  more  difficult.  A 
major  obstacle  is  the  fact  that  solutions  to  non-linear  ODEs  cannot  in  general  be  obtained  as 
closed-form  expressions,  i.e.  finite  expressions  in  terms  of  polynomials  and  elementary  functions 
such  as  exp,  sin,  cos,  In,  etc.  Hybrid  systems  with  non-linear  ODEs  are  not  at  all  uncommon  in 
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control  theory;  this  is  especially  true  of  the  class  of  piecewise-smooth  systems  (sometimes  called 
variable  structure  systems ),  which  are  used  in  the  design  of  sliding  mode  controllers  [10]. 

A  number  of  tools  and  approaches  have  been  developed  that  enable  safety  verification  of 
non-linear  systems  (e.g.  [7,  21,  36,  31,  30,  29,  24,  17,  15,  37]).  The  methods  currently  in  existence 
differ  in  a  number  of  aspects;  for  instance,  the  level  of  automation  they  provide,  the  generality 
of  system  and  inputs  specifications,  etc.  These  important  (and  at  times  subtle)  differences  make 
the  tools  difficult  to  compare  objectively.  One  approach  to  address  the  issue  could  be  to  push  for 
a  consensus  in  the  community  about  a  useful  and  fairly  general  class  of  systems  of  interest  that 
we  should  all  work  on.  However,  any  such  enterprise  would  be  necessarily  artificial  for  the  time 
being  as  there  is  no  generally  agreed-upon  classification  of  differential  equations.  In  this  work, 
we  rather  advocate  a  pragmatic  approach:  that  of  creating  a  database  of  benchmarks  that  can 
be  used  for  a  comprehensive  assessment  of  the  existing  and  future  verification  tools.  The  hope 
would  be  to  steer  the  research  towards  working  with  a  growing  set  of  examples  that  a  variety  of 
related  communities  care  about.  If  such  a  set  were  available,  a  tool  (or  an  approach)  could  easily 
be  seen  to  be  more  powerful  if  it  is  able  to  handle  (parse,  verify,  solve,  etc.)  a  larger  proportion 
of  those  examples.  Determining  which  verification  tool  is  “better”  cannot  be  entirely  objective 
as  it  would  further  need  to  take  into  account  the  tool’s  running  time  performance,  memory 
requirements,  level  of  automation,  etc.  However,  we  believe  that  the  problem  of  comparing 
verification  tools  can,  at  least  in  part,  be  addressed  by  collecting  verification  benchmarks  and 
converting  them  to  a  single  standardized  input  format.  While  this  effort  is  only  a  first  step 
towards  a  more  ambitious  goal,  we  feel  it  is  important  to  initiate  the  process  of  gathering 
interesting  verification  problems  and  making  them  available  to  the  community. 

Similar  efforts  have  been  successfully  undertaken  in  fields  such  as  automated  theorem  prov¬ 
ing  (e.g.  the  TPTP  problem  library  [3]),  SAT  solving  (where  competitions,  e.g.  [1],  have  led  to 
drastic  improvements  in  the  performance  of  SAT  solvers  in  the  last  two  decades)  and  numerical 
analysis  [39],  resulting  in  improved  quality  of  the  tools  and  their  underlying  algorithms. 

Contributions 

We  (I)  provide  a  set  of  65  safety  verification  problems  featuring  non-linear  systems,  for  all  of 
which  the  safety  property  is  known  to  hold.  Further,  we  (II)  discuss  the  current  challenges 
in  comparing  verification  tools  working  with  non-linear  continuous  dynamics  and  (Hi)  outline 
ideas  for  addressing  some  of  these  difficulties. 


2  Benchmarks 

We  have  collected  a  set  of  65  safety  verification  problems  featuring  non-linear  ODEs,  which 
we  have  gathered  from  existing  papers  treating  the  problem  of  unbounded  time  safety  verifi¬ 
cation  [24,  9,  11,  37]  and  invariant  generation  for  non-linear  systems  (e.g.  [6]).  The  problems 
we  have  collected  all  share  the  property  of  having  proofs  of  safety  that  were  obtained  using 
the  methods  presented  in  the  pertinent  papers  (or  having  proofs  that  are  immediate  from  the 
results  described  therein). 

In  general,  in  order  to  fully  state  a  safety  verification  problem,  one  requires  four  pieces  of 
information: 

1.  The  system  of  ODEs,  written  using  vector  notation  as  x  =  f(x),  where  /  :  Rn  — >•  R™. 

2.  The  mode  invariant,  denoted  H  C  Rn,  which  defines  the  region  where  the  system  may 
evolve  along  the  solution  to  the  system  of  ODEs. 
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3.  The  set  of  initial  states  X0  C  Rn. 

4.  The  set  of  unsafe  (or  forbidden)  states  Xu  C  Rn. 

Remark  Note  that  it  is  sufficient  to  consider  autonomous  ODEs,  i.e.  those  in  which  the 
right-hand  side  does  not  depend  explicitly  on  the  independent  time  variable  t,  because  one 
may  always  augment  the  system  with  t  =  1  and  treat  t  as  a  state  variable.  Furthermore,  in 
many  cases  it  is  also  sufficient  to  only  consider  polynomial  problems  because  it  is  often  possible 
to  re-cast  safety  verification  problems  with  non-polynomial  terms  to  problems  only  featuring 
polynomial  functions  (see  e.g.  [25,  28]). 

The  problem  is  to  show  that  it  is  impossible  for  the  system  to  evolve  into  a  forbidden  state 
xu  £  Xu  from  any  initial  state  x0  £  X0  by  following  the  solution  ipt(x 0)  to  the  system  of  ODEs 
x  =  f(x)  for  any  time  while  it  remains  within  the  evolution  constraint  H .  Formally,  this  may 
be  written  down  as 

\/t  >  0.  Mxq  £  X0.  (Vt  e  [0,t],  ipT(x o)  £  H)  ->•  ipt(x o)  £  Xu. 

In  bounded-time  safety  verification  one  is  only  interested  in  showing  safety  up  to  some  finite 
time  bound  T  >  0,  i.e. 

Vt  e  [0,  T\.  Vx0  £  X0.  (Vr  e  [0  ,t].  <pT(xo)  £jf)4  ipt(x  o)  xu. 

Clearly,  if  the  safety  property  holds  for  unbounded  time,  it  is  guaranteed  for  any  fi¬ 
nite  time  bound,  but  not  conversely.  Since  all  the  problems  we  have  gathered  are  non¬ 
linear  and  have  proofs  of  unbounded-time  safety,  we  may  designate  this  class  of  problems 
NONLIN-UNBOUND-TIME-SAFE  in  order  to  distinguish  it  from  other  classes  of  problems  that 
we  may  wish  to  add  later  on,  such  as  e.g.  provably  safe  linear  systems,  or  provably  un¬ 
safe  systems,  etc.  In  this  section  we  will  illustrate  some  of  the  safety  verification  prob¬ 
lems  featuring  2-dimensional  ODEs.  The  full  set  of  the  65  problems  is  available  from 
http : //verivital . com/hyst/benchmark-nonlinear/ 

Example  2.1  (Non-linear  example  [9]).  Dai  et  al.  in  [9]  studied  safety  verification  using  barrier 
certificates,  illustrating  their  approach  using  the  following  system: 


x  =  2x  —  xy, 
y  =  2x2-  y. 


>, 


x 

Figure  1:  Non-linear  system  in  the  safety  verification  problem  from  [9]. 
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The  set  of  initial  states  is  given  by  x2  +  (y  +  2) 2  <  1  and  the  set  of  unsafe  states  is 
x2  +  (y  ~  1)  2  <  i§o  (shown  in  green  and  red  respectively  in  Fig.  1).  The  evolution  constraint 
is  taken  to  be  the  real  plane  R2. 

Example  2.2  (FitzHugh-Nagumo  system  example  [6]).  Ben  Sassi  et  al.  [6]  reported  a  method 
for  generating  polyhedral  invariants  for  polynomial  ODEs  and  applied  it  to  the  FitzHugh- 
Nagumo  system: 


x3 

x  =  +x-y  + 


7 

8’ 


With  the  knowledge  of  the  invariant,  by  considering  initial  states  that  lie  inside  the  invariant, 


>, 


x 

Figure  2:  Safety  verification  in  the  FitzHugh-Nagumo  system. 

e.g.  —  1  <  x  <  —0.5  A  1  <  y  <  1.5  and  letting  —2.5  <  x  <  —2  A  —2  <  y  <  —1.5  represent 
the  forbidden  states,  all  of  which  lie  entirely  outside  the  invariant,  one  may  conclude  the  safety 
property.  Fig  2  shows  the  phase  portrait  along  with  the  initial  and  the  unsafe  states  (in  green 
and  red,  respectively). 

Example  2.3  ([37],  ODE  from  [12],  Ex.  10.15  (i)).  In  previous  work  [37],  a  non-linear  ODE 
from  a  textbook  on  the  qualitative  theory  of  planar  ODEs  [12] 
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x  =  — 42x7  +  68 x6y  —  4  6x5y  +  258 x4y  +  156a :3y  +  50a :2y  +  20  xye  —  8  y7 , 
y  =  y  (11 10a:6  -  220x5y  -  3182x4y  +  478a;  V  +  487a:  V  -  102a ;y5  -  12 y6)  , 

was  used  to  create  a  safety  verification  problem  where  the  initial  states  are  given  by 
x>— lAx<— |a?/<  §  A  y  >  1  and  the  forbidden  states  satisfy  the  inequality  x  >  y  +  1 
(shown  respectively  in  green  and  red  in  Fig.  3). 


2.1  Problem  format 

We  have  chosen  to  store  our  verification  problems  in  a  format  used  by  the  SpaceEx  verification 
tool  for  hybrid  systems  [14].  While  SpaceEx  currently  cannot  work  with  non-linear  differential 
equations,  its  input  format  is  sufficiently  simple  and  convenient.  A  given  problem  in  this  format 
is  stored  in  two  separate  files 

1.  An  .xml  file  storing  the  ODE  x  =  f(x)  and  the  mode  invariant  H  of  the  system. 

2.  A  .  cf  g  file  detailing  the  initial  set  X0  and  the  set  of  forbidden  states  Xu. 

For  example,  the  verification  problem  described  in  Example  2.2,  may  be  stored  in  the  two  files 
shown  in  Fig.  4  and  Fig.  5. 


<?xml  version3" 1 . 0"  encoding3" iso -8859 -1 "? > 

<sspaceex  xmlns="http ://www-verimag. imag.fr/xml-namespaces/sspaceex"  version="0 . 2"  math=" 
SpaceEx " > 

<component  id="f itzhugh_nagumo_ben_sass i_girard_2 " > 

<param  name="x"  type="real"  local =" f alse "  dl="l"  d2="l"  dynamics =" any "/> 

<param  name="y"  type="real"  local =" f alse "  dl="l"  d2="l"  dynamics =" any "/> 

<location  id="l"  name="p"> 

<invariant  >true  </ invariant  > 

<flow>x  ,==7/8+x-x~3/3-y  &amp  ;  y,==(2*(7/10+x-(4*y)/5))/25</flow> 

</locat ion > 

</ component  > 

</ sspaceex  > 


Figure  4:  FitzHugh-Nagumo  system  dynamics,  illustrated  in  Fig.  2. 


system  =  f itzhugh_nagumo_ben_sassi_girard_2 

initially  =  " - 1 <-x  &  x <- -0 . 5  &  1 <  =  y  &  y<  =  1.5" 

forbidden  =  " -2 . 5 <  =  x  &  x<=-2  &  -2<  =  y  &  y <= - 1 . 5 

output -variables  =  x,y 

scenario  =  stc 

directions  =  box 

set -aggregation  =  "none" 

sampling -time  =  0.5 

flowpipe -tolerance  =  0.25 

time-horizon  =  9 

iter-max  =  4 

output  -  f  ormat  =  GEN 

verbosity  =  m 

output -error  =  0.001 

rel-err  =  1.0e-12 

abs-err  =  1.0e-15 


Figure  5:  SpaceEx  configuration  file  specifying  the  initial  and  forbidden  states. 
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3  Challenges 

In  using  any  significantly  broad  set  of  verification  benchmarks,  one  faces  a  number  of  challenges 
if  one  wishes  to  use  them  to  compare  safety  verification  methods  and  tools.  Firstly,  in  contrast 
to  the  world  of  SAT/SMT  solving  or  automated  theorem  proving,  verification  of  continuous 
systems  has  not  matured  to  the  point  where  the  community  has  agreed  upon  an  input  standard 
that  can  be  used  to  exchange  problems  (such  as  SMT-LIB  [2]  or  TPTP  [3]).  Also,  unlike 
with  numerical  analysis  or  simulation,  general  safety  verification  problems  need  not  have  point 
initial  conditions,  but  rather  a  set  of  initial  states  that  may  be  uncountably  infinite,  and  not 
necessarily  “nice”  (e.g.  may  be  disconnected,  non-convex,  unbounded,  etc.).  Below  we  outline 
some  important  challenges  that  stand  in  the  way  of  benchmarking  existing  verification  tools. 

•  Tools  for  bounded-time  safety  verification  based  on  computing  flowpipes  enclosing  reach¬ 
able  sets  of  non-linear  ODEs,  such  as  e.g.  Flow*,  are  often  limited  in  the  nature  of  the 
initial  and  the  forbidden  sets  of  states.  In  particular,  the  underlying  algorithms  used  in 
these  tools  require  the  set  of  initial  states  to  be  bounded  (unlike  in  Example  2.3);  ideally 
given  by  a  hyper-rectangle  (unlike  Example  2.1).  On  the  other  hand,  methods  for  auto¬ 
matic  unbounded-time  safety  verification  based  on  searching  for  appropriate  continuous 
invariants  (e.g.  [29,  37])  are  capable  of  working  with  much  broader  classes  of  initial  and 
forbidden  regions.  For  instance,  semi-algebraic  initial  regions  that  are  unbounded,  non- 
convex,  or  whose  description  features  a  combination  of  conjunctions  and  disjunctions  do 
not  present  a  problem. 

Remark  At  the  same  time,  tools  based  on  flowpipe  construction  can  sometimes  give  a 
sense  of  the  “hardness”  of  the  verification  problem  when  they  fail  to  prove  safety  up  to 
some  given  time  bound,  whereas  invariant-based  verification  tools  typically  do  not  provide 
useful  insights  into  the  nature  or  the  difficulty  of  the  problem  when  they  fail. 

•  Tools  that  employ  interval  arithmetic  often  require  bounds  on  the  state  variables  of  the 
system  (e.g.  HSolver  [33,  34],  dReach  [21]),  which  technically  renders  them  inapplicable 
to  safety  verification  problems  where  the  evolution  constraint  H  is  unbounded,  e.g.  given 
by  Rn. 

•  Certain  tools  (e.g.  Flow*)  cannot  work  with  sets  described  by  strict  inequalities  (such  as 
the  forbidden  states  in  Example  2.3).  While  it  would  be  sound  to  simply  over-approximate 
the  closure  of  such  sets  by  relaxing  the  inequalities  to  be  non-strict,  this  step  currently 
needs  to  be  performed  manually  by  the  user  and  (inevitably)  affects  the  reachability  anal¬ 
ysis. 

•  The  performance  of  tools  often  depends  heavily  on  the  user-specified  options,  such  as  e.g. 
the  fixed/adaptive  time  steps  used  for  the  verified  integration,  error  tolerances,  etc.  It  is 
presently  not  apparent  how  one  might  automatically  translate  “good”  settings  from  one 
verification  tool  to  another,  or  indeed  automatically  arrive  at  good  settings  for  a  particular 
tool  in  the  first  place.  Thus,  some  verification  tools  that  are  designed  to  be  fully  automatic 
rely  crucially  on  the  user  choosing  the  right  settings,  which  is  typically  difficult  for  a  non¬ 
expert. 

•  Some  unbounded-time  verification  methods  (e.g.  [31])  likewise  require  significant  manual 
input  from  the  user,  such  as  e.g.  selecting  templates  for  polynomial  functions.  It  is  yet 
unclear  how  these  methods  can  be  meaningfully  compared  to  methods  that  provide  a 
greater  level  of  automation. 

•  Uncertainty  in  the  continuous  dynamics  is  permitted  by  some  verification  tools  (e.g.  Flow*), 
but  not  others. 
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4  Outlook 

Safety  verification  problems  for  non-linear  systems  are  very  useful  for  assessing  the  utility  and 
efficiency  of  invariant  generation  methods  (e.g.  [35,  38,  6,  29,  26,  40,  17,  37]),  as  well  as  tools 
based  on  verified  integration  of  ODEs  (e.g  [7,  27,  20,  21]).  We  are  hopeful  that  maintaining  and 
further  populating  the  set  of  verification  benchmarks  will  result  in  improvements  to  the  exist¬ 
ing  capabilities  offered  by  the  tools  for  both  bounded  and  unbounded-time  safety  verification. 
Improvements  in  invariant  generation  would  also  greatly  benefit  deductive  verification  tools  for 
hybrid  systems,  such  as  theorem  provers  (e.g.  [30,  15,  23]). 

At  least  some  of  the  challenges  outlined  in  the  previous  section  can  potentially  be  addressed 
using  HyST  [5],  a  source  transformation  tool  for  hybrid  systems  that  takes  as  input  a  hybrid 
system  verification  problem  in  the  SpaceEx  format  and  translates  it  into  formats  accepted  by 
other  verification  tools.  In  addition  to  translating  between  the  various  problem  formats,  HyST 
is  able  to  work  with  its  internal  representation  of  the  verification  problem  through  so-called 
model  transformation  passes,  which  can  address  issues  that  affect  particular  verification  tools. 
For  instance,  currently  HyST  can  add  identity  reset  maps  to  transitions  in  hybrid  automata, 
split  transition  guards  with  disjunctions,  etc.  A  potentially  interesting  future  transformation 
pass  could  be  implemented  in  HyST  to  convert  continuous  systems  with  uncertainty  into  hybrid 
systems  in  which  there  is  no  uncertainty  in  the  continuous  dynamics,  e.g.  following  the  work 
of  Ramdani  et  al.  [32] . 

At  present,  HyST  can  translate  problems  into  formats  accepted  by  Flow*,  dReach,  HyCre- 
ate  [4] ,  HyComp  [8]  and  SpaceEx.  An  interesting  future  direction  would  be  to  extend  it  to  also 
work  with  invariant  generation  tools  and  add  model  transformation  passes  to  soundly  convert 
safety  verification  problems  that  currently  cannot  be  processed  by  some  of  the  verification  tools 
into  a  form  that  is  amenable  to  analysis. 

In  collecting  safety  verification  benchmarks  it  is  profitable  to  find  a  useful  classification.  One 
could  separate  verification  problems  for  continuous  systems  into  classes  depending  on  certain 
features,  such  as: 

•  the  type  of  continuous  dynamics,  e.g.  constant /linear /non- linear, 

•  the  dimensionality  of  the  system  (i.e.  the  number  of  state  variables,  |®|), 

•  the  type  of  safety  verification  (i.e.  bounded  versus  unbounded  time), 

•  the  nature  of  the  evolution  constraint  (e.g.  bounded  versus  unbounded  state  space), 

•  the  nature  of  the  initial  and  forbidden  set  (bounded  versus  unbounded;  if  bounded,  hyper¬ 
rectangles  versus  more  general  sets),  and 

•  the  nature  of  the  verification  problem  itself  (i.e.  is  the  system  safe  or  unsafe?). 

Such  a  classification  will  certainly  become  important  in  the  future  as  more  verification  problems 
are  gathered  and  added  to  our  collection.  Our  initial  set  of  65  problems  (which  we  tentatively 
labelled  NONLIN-UNBOUND-TIME-SAFE)  belongs  to  one  of  the  most  general  classes  under  this 
scheme,  since  it  makes  few  assumptions  about  the  nature  of  the  verification  problem.  This 
generality  makes  it  difficult  to  use  the  problems  for  benchmarking  existing  tools,  but  at  the 
same  time  serves  to  bring  out  their  current  limitations. 
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Abstract.  We  investigate  decoupling  abstractions,  by  which  we  seek  to 
simulate  (i.e.  abstract)  a  given  system  of  ordinary  differential  equations 
(ODEs)  by  another  system  that  features  completely  independent  (i.e. 
uncoupled)  sub-systems,  which  can  be  considered  as  separate  systems  in 
their  own  right.  Beyond  a  purely  mathematical  interest  as  a  tool  for  the 
qualitative  analysis  of  ODEs,  decoupling  can  be  applied  to  verification 
problems  arising  in  the  fields  of  control  and  hybrid  systems.  Existing  ver¬ 
ification  technology  often  scales  poorly  with  dimension.  Thus,  reducing  a 
verification  problem  to  a  number  of  independent  verification  problems  for 
systems  of  smaller  dimension  may  enable  one  to  prove  properties  that  are 
otherwise  seen  as  too  difficult.  We  show  an  interesting  correspondence 
between  Darboux  polynomials  and  decoupling  simulating  abstractions 
of  systems  of  polynomial  ODEs  and  give  a  constructive  procedure  for 
automatically  computing  the  latter. 

Keywords:  ordinary  differential  equations,  Darboux  polynomials,  sim¬ 
ulation,  abstraction,  decoupling 


1  Introduction 

Simulation  relations  are  an  important  concept  in  the  study  of  both  discrete  and 
continuous  dynamical  systems.  Informally  speaking,  a  system  simulates  another 
system  if  it  over-approximates  its  set  of  possible  behaviours.  In  practice,  when 
analyzing  systems,  one  often  wants  to  construct  simulations  of  the  original  sys¬ 
tem  that  are  in  some  sense  “simpler”  to  analyze.  Then,  by  demonstrating  some 
property  of  interest  in  the  simulation  one  may  infer  the  property  in  the  original 
system. 

In  [H]  Sankaranarayanan  investigated  an  interesting  technique  for  construct¬ 
ing  simulations  of  continuous  systems  by  employing  change  of  basis  transforma¬ 
tions.  It  was  shown  how  linearizing  change  of  basis  transformations  of  non-linear 

*  This  work  was  supported  by  the  Air  Force  Research  Laboratory  (AFRL)  through 
contract  number  FA8750-15-1-0105  and  the  Air  Force  Office  of  Scientific  Research 
(AFOSR)  under  contract  numbers  FA9550-15-1-0258  and  FA9550-16-1-0246. 
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systems  of  ODEs  can  yield  simulations  in  which  the  dynamics  is  given  by  a  sys¬ 
tem  of  linear  ODEs.  The  motivation  for  considering  such  transformations  is  clear, 
since  linear  systems  cannot  exhibit  some  of  the  rich  dynamic  phenomena  found 
in  their  non-linear  counterparts  and  are  more  amenable  to  analysis  nn.  In  this 
paper  we  consider  simulations  of  non-linear  ODEs  of  a  different  kind:  instead  of 
linear  dynamics,  we  seek  to  construct  simulations  that  are  potentially  non-linear, 
but  whose  analysis  can  be  performed  in  a  lower-dimensional  space  than  that  of 
the  original  system. 

Although  our  focus  in  this  paper  is  on  analyzing  purely  continuous  systems, 
the  methods  we  present  are  motivated  by  the  broader  goal  of  aiding  the  task 
of  automatic  verification  of  hybrid  dynamical  systems  whose  continuous  modes 
are  governed  by  non-linear  ODEs.  Hybrid  systems  combine  discrete  and  contin¬ 
uous  behaviour;  their  formal  modelling  and  verification  is  of  increasing  interest 
and  importance  to  modern  engineering,  where  discrete  digital  controllers  are 
used  to  control  continuously  evolving  physical  plants.  In  recent  years,  verifica¬ 
tion  technology  for  hybrid  systems  has  seen  significant  advances  and  a  number 
of  interesting  case  studies  have  been  reported,  e.g.  verification  of  train  control 
systems  mm\.  aircraft  collision  avoidance  protocols  m,  descent  guidance 
control  software  in  a  lunar  lander  |2Bj  and  satellite  rendezvous  manoeuvres  El, 
to  give  a  few  examples.  However,  non-linear  ODEs  appearing  in  hybrid  system 
models  often  present  a  serious  challenge  to  verification  due  to  their  inherent 
complexity.  In  this  paper  we  seek  to  overcome  some  aspects  of  this  hurdle  by 
constructing  simulations  of  non-linear  ODEs  with  structure  that  more  readily 
lends  itself  to  analysis. 

1.1  Contributions 

In  this  paper  we  (I)  define  decoupled  simulating  abstractions  of  non-linear  ODEs, 
discuss  their  utility  and  relationship  to  first  integrals  El  and  constant-scale 
continuous  consecutions  l23| .  (II)  We  give  an  algorithm  for  checking  whether  a 
given  set  of  polynomial  abstract  basis  functions  can  be  used  to  create  a  decou¬ 
pled  abstraction  of  a  system  of  polynomial  ODEs  and  then  (III)  employ  the 
theory  of  Darboux  polynomials  El  to  give  sufficient  criteria  for  non-existence  of 
polynomial  abstract  basis  functions  suitable  for  constructing  decoupled  polyno¬ 
mial  abstractions.  Lastly,  (IV)  we  show  how  Darboux  polynomials  can  be  used 
to  construct  the  abstract  basis  functions  for  decoupled  abstractions  whenever 
they  exist.  We  conclude  with  a  summary  of  our  findings,  an  overview  of  related 
work  and  directions  for  future  research. 

1.2  Preliminaries 

An  autonomous  n-dimensional  system  of  ODEs  has  the  following  form: 

=  fi(x1,x2,  ■  ■  -,xn), 

&n  —  fn(x  1,^2,  ■  •  •  ,  Xn  ) , 
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where  for  i  G  {1, . . . ,  n}  each  :  Rra  — »•  R  is  a  real- valued  function  (typically  C1), 
and  ±i  denotes  the  time  derivative  of  xi}  i.e.  4zXi(t).  In  applications,  constraints 
are  often  imposed  on  the  states  where  the  system  is  allowed  to  evolve,  i.e.  the 
system  may  only  evolve  inside  some  given  set  H  C  R™,  which  is  known  as  the 
evolution  constraint.  We  may  write  this  more  concisely  using  vector  notation 
as  x  =  f{x),  x  G  H.  Here  x  =  (aq, . . . ,  xn)  and  /  :  Rn  — *  R"  is  a  vector  field 
generated  by  the  system,  i.e.  f(x)  =  (fi(x), . . . ,  fn(x))  for  all  x  G  Rn.  When  no 
evolution  constraint  is  specified,  H  is  assumed  to  be  R™. 

A  solution  to  the  initial  value  problem  for  the  system  of  ODEs  x  =  f(x) 
with  initial  value  Xq  G  R"  is  a  differentiable  function  ipt(x o)  :  (a,  6)  — >  R" 
defined  for  all  t  within  some  non-empty  extended  real  interval  including  zero,  i.e. 
t  G  (a,  b )  C  Ru{oo,  —  oo},  where  a  <  0  <  b,  and  such  that  o)  =  f{tpt{x o)) 

for  all  t  G  (a,  b).  If  the  solution  ipt(x 0)  is  available  in  closed- formFl then  one  can 
answer  questions  about  the  temporal  behaviour  of  the  system  (such  as  e.g.  safety 
and  liveness)  by  analyzing  the  closed-form  expression.  In  practice,  however,  it 
has  long  been  established  that  explicit  closed-form  solutions  to  non-linear  ODEs 
are  highly  uncommon  m- 

In  this  paper  we  will  work  with  systems  of  ODEs  whose  right-hand  sides 
are  given  by  polynomials  in  the  state  variables  x%, . . . ,  xn.  Formally,  we  say  that 
fi  G  R[Ai, . . . ,  Xn]  for  all  i  G  {1, ... ,  n},  where  R[Xi, . . . ,  Xn]  denotes  the  ring  of 
multivariate  polynomials  with  real  coefficients  and  indeterminates  Xi, . . . ,  Xn . 
We  write  fi(xi, . . .  ,i„)  when  we  wish  to  make  it  clear  that  the  polynomial  is 
treated  as  a  function,  with  indeterminates  replaced  by  the  appropriate  variables. 
Polynomial  systems  of  ODEs  are  necessarily  locally  Lipschitz  continuous,  which 
guarantees  existence  of  unique  solutions  on  some  non-trivial  time  interval  for 
any  initial  value  *0  G  Rn  (by  the  Picard-Lindelof  theorem;  see  e.g.  [27]). 

1.3  Coupling 

Given  a  system  of  ODEs  x  =  f(x ),  the  maximum  coupling  coefficient  (henceforth 
mcc)  is  the  size  of  the  largest  sub-system  with  no  independent  sub-systems.  To 
define  rigorously,  we  construct  a  finite  coupling  graph  CG  =  (V,  E).  where  the 
set  of  vertices  is  precisely  the  set  of  state  variables,  i.e.  V  =  {x\, . . .  ,xn},  and 
there  is  an  edge  from  ij  to  some  other  vertex  Xj,  i.e.  ( Xi,Xj )  G  E  with  i  7^  j,  if 
and  only  if  7^  0.  The  coupling  coefficients  cc  are  a  finite  multiset  of  natural 
numbers  corresponding  to  the  orders  (i.e.  the  numbers  of  vertices)  of  all  the 
weakly  connected  components  in  CG.  The  coefficient  mcc  is  defined  to  be  the 
maximum  order  of  the  weakly  connected  components  in  CG,  i.e.  mcc  =  maxcc. 

Definition  1  (Uncoupled  system).  A  system  of  ODEs  x  =  f(x)  is  uncou¬ 
pled  if  and  only  if  its  mcc  =  1,  i.e.  if  the  rate  of  change  of  each  state  variable  is 
completely  independent  of  the  other  variables^ 

1  By  this  we  understand  a  finite  expression  in  terms  of  polynomials  and  elementary 
functions  such  as  sin,  cos,  exp,  In,  etc. 

4  An  equivalent  (but  less  flexible)  definition  would  state  that  a  system  x  =  f(x)  is 
uncoupled  if  and  only  if  the  Jacobian  matrix  J/  is  diagonal. 
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Example  1.  Consider  the  following  two  planar  polynomial  systems: 

Xi  =  x\x2  +  5xi  —  1,  x\  =  x3  +  5a;  i  —  10, 

x-2  =  3x3  +  2xiX2  —  Xi.  ±2  =  2x2  +  3x2  +  1. 

The  system  on  the  left  has  mcc  =  2  because  the  vertices  {xi,  X2}  in  the  coupling 
graph  have  edges  connecting  them  in  both  directions,  since  g^-(xfx2  +  5xi  — 1)  = 
xf  /  0  and  ^  (3x3  +  2xiX2  —  xi)  =  2x2  —  1  /  0.  On  the  other  hand,  the  system 
on  the  right  has  mcc  =  1  (i.e.  is  uncoupled)  because  (x3  +  5xi  —  10)  =  0 
and  g|/2x2  +  3x2  +  1)  =  0  and  therefore  the  vertices  {xi,  X2}  in  the  graph  are 
disconnected. 


Uncoupled  systems  are  appealing  first  and  foremost  because  their  1- 
dimensional  sub-systems  can  be  analyzed  independently,  following  a  standard 
technique  for  1-dimensional  flows  (see  e.g.  [2SJ  Chapter  2]).  For  instance,  con¬ 
sider  the  1-dimensional  system  x  =  x3  +  5x2  +  x  —  10.  This  system  evolves  on 
the  real  line  and  has  fixed  points  at  the  real  roots  of  x3  +  5x2  +  x  —  10,  of  which 
there  are  three:  {—2,  \  (—3  —  \/29)  ,  \  (—3  +  a/29)  } .  The  direction  of  the  flow  is 
to  the  right  whenever  the  graph  of  x  is  above  zero  (i.e.  the  rate  of  change  of  x 
is  positive)  and  to  the  left  when  it  is  below  (the  rate  of  change  is  negative),  as 


shown  in  Figure  1.3 


Fig.  1.  Analysis  of  the  1-dimensional  system  x  =  Xs  +  5x2  +  x  —  10. 


From  inspecting  the  figure,  one  can  readily  see  how  one  can  construct  the  set 
of  reachable  states  of  any  given  initial  point  xo  in  a  1-dimensional  polynomial 
system  x  =  f(x):  either  the  point  is  a  root  of  the  right-hand  side,  i.e.  /(x 0)  =  0, 
in  which  case  Xq  remains  invariant  and  the  reachable  set  is  simply  {xo},  or  xo 
is  not  a  root,  i.e.  /(x)  /  0,  in  which  case  the  reachable  set  is  an  interval  of  the 
form  [xo,  r)  or  (r,  x’o],  where  r  £  RU  {00,  —00}  is  either  a  real  root  of  /  or  it  is  00 
or  —00,  respectively  (if  there  are  no  real  roots  in  the  direction  of  motion).  The 
reachable  set  from  any  initial  point  Xq  £  R"  in  a  uncoupled  system  can  thus  also 
be  bounded  by  combining  the  independent  reachable  sets  in  the  1-dinrensional 
sub-systems. 


4 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

245 


Bounded-time  reachable  set  computation  using  verified  integration  methods 
is  also  made  easier  because  large  systems  of  non-linear  ODEs  are  typically  ex¬ 
pensive  to  integrate  using  methods  that  yield  tight  enclosures  [TB]  (such  as  Tay¬ 
lor  models  urn  whereas  in  an  uncoupled  system,  no  matter  how  large,  each 
1-dimensional  sub-system  can  be  integrated  separately.  An  enclosure  of  the  so¬ 
lution  to  the  whole  system  at  some  time  t  can  then  be  constructed  directly  from 
the  enclosures  of  the  solutions  to  the  sub-systems  at  that  time. 


2  Decoupled  Simulating  Abstractions 


In  what  follows,  we  will  adopt  the  approach  described  by  Sankaranarayanan 
in  [22]  to  define  simulating  abstractions  of  non-linear  ODEs  using  appropriate 
change  of  basis  transformations. 

Definition  2  (Simulating  abstraction).  For  a  system  x  =  f(x),  x  £  H, 
where  f  :  R™  — >■  R"  is  locally  Lipschitz  continuous,  equipped  with  an  initial  set 
of  states  Xo  C  Rn,  a  system  a  =  G(a),  a  £  H ,  where  G  :  Rm  — >  Rm  is  locally 
Lipschitz  continuous  and  equipped  with  an  initial  set  of  states  Xo  C  Rm  is  a 
simulating  abstraction  if  there  exists  a  smooth  (i.e.  C°°)  mapping  a  :  Rn  — >  Rm 
such  that:  (i)  a(X o)  C  Xo,  (ii)  a(H)  C  H,  and  (Hi)  for  any  trajectory  (i.e. 
solution  in  non-negative  time)  <pr(x Q)  :  [0,T)  — >  H  of  the  system  x  =  f{x),  x  £ 
H,  the  trajectory  a  o  ipT(x 0)  :  [0,T)  — >•  H  is  a  trajectory  of  a  =  G(a),  a  £  H. 

To  ensure  that  the  last  condition  in  the  above  definition  holds,  it  is  sufficient  to 
show  that  G(a(x))  =  Ja-  f(x),  where  Ja  is  the  Jacobian  of  the  smooth  mapping 
a  w.r.t.  the  state  variables  x\, . . .  ,xn  (see  [221  Theorem  2.1]),  i.e. 

(da.  i  da\ 

dx\  ’  *  ’  dxn 

\  ■■.  : 

dam  dam 

dx i  ’  *  ’  dxn 

Definition  3  (Lie  derivative).  For  a  given  system  of  ODEs  x  =  f(x),  the 
Lie  derivative  of  a  smooth  function  p  :  R"  — >  R  is  given  by 

n  F) 

£f(p)  =  Vp  •  /  =  ^  ^  •  ft. 

i—1  1 

Note  that  since  fi(x)  =  £/(p)  =  '  lit)  =  iu  ^ 0 ^  deriva¬ 

tive  of  the  function  p  with  respect  to  time,  which  we  denote  by  p. 


Let  us  recall  that  the  gradient  Vp  gives  the  vector  of  all  the  partial  derivatives 
of  p,  i.e.  Vp  =  . . . ,  ,  and  thus  the  necessary  condition  for  (iii)  in 

Definition  [2]  to  be  satisfied  may  be  equivalently  stated  as: 

/  Vai 

G(a)  =  :  |  •/  = 

\  Va„ 
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Remark  1.  It  is  important  to  note  that,  following  Def.[2j  solutions  to  simulating 
abstractions  are  guaranteed  to  exist  for  at  least  as  long  as  they  do  in  the  concrete 
system.  This  property  is  crucial  to  soundness  of  the  abstraction.  A  rather  differ¬ 
ent,  but  in  a  certain  sense  more  general,  concept  was  explored  by  Platzer,  who 
introduced  differential  ghosts  ng.  where  the  original  dynamics  is  augmented  by 
introducing  some  fresh  variables  whose  rate  of  change  may  feature  the  newly  in¬ 
troduced  variables  themselves,  but  is  not  restricted  in  the  same  way  as  in  Def.[2] 
However,  extra  care  needs  to  be  taken  to  ensure  that  the  solutions  of  the  newly 
defined  dynamics  exist  for  at  least  as  long  as  the  solutions  to  the  original  system 
(e.g.  see  [T51  Proof  of  Theorem  38]). 

Definition  4  (Decoupling  simulating  abstraction).  Given  a  system  of 
ODEs  x  =  f(x),  a  simulating  abstraction  a  =  G(a )  is  decoupling  if  and 
only  if  the  equalities  £f{a  i)  =  Gi(aq), . . . ,  £/(am)  =  Gm(am)  hold,  where 
(G i, . . . ,  Gm )  =  G.  Such  an  abstraction  is  thus  uncoupled: 

ai  =  Gi(ai), 


dm  —  Gm(c^m)- 

In  what  follows,  we  will  give  some  examples  of  how  first  integrals  (see  e.g.  TO]) 
and  constant-scale  continuous  consecutions  [23]  provide  the  abstract  basis  func¬ 
tions  a.  which  lead  to  decoupling  simulating  abstractions. 

Example  2  (Algebraically  integrable  system).  The  3-dimensional  system 

xi  =  xi(x3  -  x2), 
x2  =  x2{x\  -  x3), 
x3  =  x3(x2  -  Xi), 

has  two  independent  polynomial  conserved  quantities,  i.e.  first  integrals,  given 
by  oi\  =  X\X2x3  and  a2  =  X\  +  x2  +  x3  (see  ®  Ex.  75]).  If  we  let  a  =  (aq,  a2), 
we  obtain  the  decoupling  simulating  abstraction  a  =  0,  i.e.  dq  =  0,d2  =  0. 

Remark  2.  A  polynomial  system  x  =  f(x)  of  size  n  is  algebraically  integrable  if  it 
possesses  n  —  1  independent  polynomial  conserved  quantities  (also  known  as  first 
integrals;  see  HQ®),  i.e.  polynomials  {oq, . . . , an- 1 } ,  where  for  all*  =  1, ... , n— 1 
one  has  £/(cq)  =  a,  =  0.  Algebraic  integrability  is  a  very  powerful  property, 
since  it  allows  one  to  construct  tight  approximations  of  the  orbit  ^(xf),  i.e.  the 
reachable  set  from  x3  £  R™  in  positive  as  well  as  negative  time.  That  is,  for 
any  given  point  Xq  £  R",  if  one  evaluates  each  first  integral  a\, . . .  ,an-\  at 
x3,  one  obtains  real  constants  Ci, . . . ,  cn_i.  The  orbit  through  x3  is  guaranteed 
to  satisfy  the  formula  oq  =  C\  A  •  •  •  A  ara_i  =  c„_ i,  which  corresponds  to  a 
(real)  algebraic  subset  of  R"  given  by  the  common  real  roots  of  the  polynomials 
on  —  Cj.  Every  point  a3  £  R™-1  in  such  an  abstract  system  a  =  0  is  invariant 
and  corresponds  to  a  real  (and  invariant)  algebraic  set  containing  the  orbit  of 
the  system  x  =  f(x). 
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Polynomials  p  such  that  £ f(p)  =  A p  for  some  A  £  R.  generalize  polynomial 
first  integral^]  and  were  investigated  by  Sankaranarayanan  et  al.  in  |23j.  where 
they  were  used  in  constant-scale  continuous  consecution  conditions.  In  general, 
if  one  can  find  polynomials  a\, . . .  ,am  that  satisfy  £/(a,;)  =  A  iCti  ,  Ai  £  ffi.  for 
all  *  £  {1, ... ,  to},  then  one  obtains  a  decoupling  abstraction  of  the  form 


di  —  Aiai, 


We  generalize  this  idea  to  decoupling  polynomial  abstractions  by  considering 
polynomial  functions  on  £  M[Xl,  . . . ,  Xn]  such  that  £/(«i)  =  Gt(a),  where  Gi  £ 
K[Af],  i.e.  the  derivative  of  may  be  expressed  as  a  polynomial  in  with  real 
coefficients. 

Example  3  (Decoupling  simulating  abstraction) .  Consider  the  coupled  system: 

*1  =  -(1  —  3*1  +  2x\  —  6*2  +  4*i*2  +  2*|), 

O 

*2  =  X  (— 1  —  3*1  +  *1  +  2*1*2  +  x\) . 

O 

Let  ai  =  *i  +  *2  —  1,  «2  =  *i  —  2*2.  If  we  consider  a.  =  (ai,  0:2),  we  arrive  at 
the  following  system  (left),  which  can  be  expressed  as  an  uncoupled  system  in 
the  new  basis  (right): 

di  =  —2*i  +  *1  —  2*2  +  2*1*2  +  x\7  di  =  af  —  1, 

d2  =  1  +  *1  —  2*2,  «2  =  0:2  +  1  • 

3  Existence  and  Generation  of  Abstraction  Polynomials 

In  what  follows,  we  investigate  the  existence  of  polynomials  that  can  be  used 
to  construct  decoupling  simulating  abstractions  of  a  given  system.  We  show  in 
SectionphT]  that  their  existence  (to  a  given  polynomial  degree)  is  decidable  and 
give  a  sufficient  criterion  for  their  non-existence  (to  a  given  degree)  based  on  the 
existence  of  so-called  Darboux  polynomials  (e.g.  see  USD-  We  then  explore  the 
problems  of  checking  and  generation.  The  checking  problem  is  concerned  with 
determining  whether  a  given  candidate  polynomial  is  suitable  for  constructing 
a  decoupling  simulating  abstraction.  In  Section[ffi2|  we  describe  a  procedure  for 
solving  the  checking  problem.  In  Section[T3]  we  present  a  technique  for  gener¬ 
ating  all  suitable  polynomials  for  the  decoupling  abstract  basis  (up  to  a  given 
polynomial  degree). 

5  i.e.  p  is  a  first  integral  if  £/(p)  =  A p  where  A  =  0. 
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3.1  Decidability  and  Darboux  Existence  Criterion 

For  polynomial  systems  of  ODEs  x  =  f(x),  the  problem  of  finding  a  non¬ 
constant  polynomial  in  the  state  variables,  p  £  M[Xi,  . . . ,  Xn],  for  the  decou¬ 
pling  abstract  basis  reduces  to  searching  for  those  p  such  that  £f{p)  =  G(p), 
where  G  £  R[X],  i.e.  G  is  a  univariate  polynomial  with  real  coefficients.  There 
may,  however,  be  no  such  polynomial.  Fortunately,  it  is  decidable  to  check  for 
existence  of  such  a  p. 

Proposition  1  (Existence  of  decoupling  abstract  basis  polynomials). 

Given  a  positive  integer  d  and  a  polynomial  system  x  =  f(x),  it  is  decidable 
to  check  whether  there  exists  a  polynomial  p  £  M[Xi, . . . ,  Xn\  of  total  degree  d 
such  that  £f(p)  =  G(p),  where  G  £  R[X]  is  a  univariate  polynomial  with  real 
coefficients. 

Proof.  The  problem  can  be  stated  as  a  sentence  in  the  theory  of  real  arithmetic 
which  is  decidable  [2fi].  Let  Ao,...,Afc  denote  the  unknown  coefficients  of  the 
generic  polynomial  template  p  of  degree  d,  where  k  :=  —  1  is  the  number 

of  non-constant  monomials  of  degree  at  most  d  in  n  variables.  The  Lie  derivative 
£f{p)  can  therefore  be  symbolically  computed  (Def.|3|.  Let  k0,  ...  ,nm  denote  the 
unknown  coefficients  of  the  polynomial  G  £  R[X]  where  m  :=  [deg (£/(p))/d~|. 
The  decision  problem  stated  in  the  proposition  is  therefore  equivalent  to  deciding 
the  truth  of  the  following  sentence: 

3  (A0, . . . ,  Afc)  G  Rfc+1.  3  K...,Mer+1. 

V(3fi, . . . ,  X.nf  £  R”.  d  >  0  A  £/{p)  —  (ko  T  ki p  -(-••■  3-  nmpm)  =  0  . 

If  Ao  denotes  the  constant  term  of  the  generic  polynomial  template  p ,  then  the 
condition  d  >  0  is  equivalent  (over  the  reals)  to  the  inequality  )T)0< -<fc  >  0; 
ensuring  that  p  is  non-constant.  □ 

In  practice,  there  is  currently  no  question  of  applying  existing  decision  pro¬ 
cedures  to  formulas  constructed  in  the  proof  or  Prop. [I]  The  complexity  of  the 
most  popular  procedure  for  real  quantifier  elimination  (CAD,  due  to  Collins  [4]) 
is  doubly  exponential  in  the  number  of  variables.  In  Section[X3|  we  will  pursue  a 
more  promising  method  of  searching  for  decoupling  abstract  basis  polynomials. 
First,  we  shall  recall  so-called  Darboux  polynomials ,  a  well-known  tool  in  the 
study  integrability  of  dynamical  systems  (e.g.  see  uni),  and  use  them  to  give 
a  non-existence  criterion  for  decoupling  abstract  basis  polynomials.  We  then 
explore  an  interesting  relationship  between  the  two  concepts. 

Definitions  (Darboux  polynomial).  A  polynomial  q  £  K\X i,...,Xn], 
where  K  is  a  field  of  characteristic  zero  (e.g.  C,R,  Qj,  is  a  Darboux  polyno- 
mia  f\for  x  =  /( x)  iff  £f(q)  =  A  q,  for  some  A  €  K[X1, . . . ,  Xn\. 

6  When  q  is  a  constant,  the  Darboux  polynomial  is  trivial [TOl  Definition  2.14],  In  this 
paper  we  will  generally  be  interested  in  the  non-trivial  case. 

8 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

249 


Proposition  2  (Criterion  for  non-existence  of  decoupled  abstractions). 

If  a  given  system  x  =  f{x)  does  not  admit  any  Darboux  polynomials  over  C  of 
degree  d,  then  there  is  no  polynomial  p  £  R[Xl,  . . . ,  Xn\  of  degree  d  such  that 
£/{p)  =  G(p)  for  some  non-constant  G  £  M[X]. 

Proof.  We  prove  the  contrapositive.  Suppose  there  exists  a  polynomial  p  £ 
R[Ai, . . . ,  Xn\  such  that  £/(p)  =  G(p),  where  G  £  R[X]  is  non-constant.  By 
the  fundamental  theorem  of  algebra,  G  must  have  at  least  one  complex  root 
c  £  C.  Therefore  G  =  (X  —  c)H ,  where  H  £  C[X].  We  see  that  (jp  —  c)  is  a 
Darboux  polynomial  for  the  system  because 

£/(p  -  c)  =  £/(p)  -  £/(c)  =  £/(p)  =  G(p)  =  (p-  c)H(p). 

The  degree  of  the  Darboux  polynomial  p  —  c  is  equal  to  the  degree  of  p.  □ 

3.2  Checking  Abstraction  Polynomial  Candidates 

Before  proceeding  to  methods  for  generating  decoupling  abstract  basis  polynomi¬ 
als  for  polynomial  systems  x  =  f(x),  we  discuss  the  (easier)  problem  of  checking 
if  for  a  given  p  £  M[Xi, . . .  ,Xn]  one  can  write  £/(p)  =  G{p),  where  G  £  M[X]. 

In  general,  given  any  two  polynomials  P,p  £  M[A'1; . . . ,  Xn],  if  deg(P)  > 
deg(p),  one  may  obtain  a  rewriting  P  =  G(p)  by  solving  a  system  of  lin¬ 
ear  equations.  One  proceeds  by  first  defining  the  maximum  degree  of  a  pos¬ 
sible  G  to  be  d  =  |~deg(P)/deg(p)] .  If  an  appropriate  rewriting  exists,  then 
there  is  guaranteed  to  be  a  solution  (Ao,...,Ad)  £  Md+1  to  the  equation 
P  =  Xq  +  Xip  +  X2P2  +  •  •  •  +  A dPd-  By  expanding  and  equating  the  monomial 
coefficients  on  both  sides  one  arrives  at  a  system  of  linear  equations  (of  size 
no  larger  than  the  number  of  monomials  of  P)  in  the  real  variables  Ao, . . . ,  \d- 
Thus,  in  the  worst  case,  one  has  to  solve  a  linear  system  with  d+ 1  variables  and 
("dc ecLuati°nal  constraints.  A  solution  may  be  computed  using  a  linear 
solver  and  the  rewriting  polynomial  constructed  as  G  =  A0  +  AiX  +  •  •  •  +  A dXd. 
In  what  follows,  we  will  refer  to  the  procedure  for  obtaining  the  rewriting  as 
Rewrite,  that  is  Rewrite(P,p)  gives  G  whenever  P  =  G{p). 

Remark  3.  It  is  worth  remarking  that  the  procedure  Rewrite  can  be  imple¬ 
mented  by  performing  successive  polynomial  reductions ,  rather  than  by  solving 
a  linear  program.  Polynomial  reduction  extends  polynomial  division  for  univari¬ 
ate  polynomials  to  the  multivariate  case  and  in  general  requires  the  computation 
of  Grobner  bases.  This  functionality  is  available  in  most  modern  computer  alge¬ 
bra  systems. 

3.3  Automated  Generation  of  Decoupling  Abstractions 

A  highly  efficient  method  for  synthesizing  polynomial  first  integrals  for  polyno¬ 
mial  ODEs  was  reported  by  Matringe  et  al.  in  HE  where  the  synthesis  problem 
is  reduced  to  computing  the  null  space  of  a  matrix  with  real  entries.  In  [7],  the 
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authors  extended  the  work  of  Matringe  et  al.  to  generate  real  algebraic  invari¬ 
ants  of  polynomial  ODEs,  giving  a  search  procedure  for  the  most  general  class 
of  invariant  sets  that  can  be  expressed  using  polynomial  equations.  The  same 
procedure  can  be  used  to  generate  Darboux  polynomials  over  the  reals  or  over 
the  complexes  only  by  changing  the  underlying  computational  field.  In  general, 
there  is  no  known  bound  for  the  degree  of  Darboux  polynomials  in  a  given  sys¬ 
tem.  However,  the  automatic  generation  procedure  is  guaranteed  to  find  all  the 
independent  Darboux  polynomials  for  the  system  up  to  a  given  degree. 

In  this  section,  we  explore  the  relationship  between  polynomials  in  a  decou¬ 
pling  abstract  basis  and  Darboux  polynomials.  This  relationship  will  enable  us 
to  exploit  the  efficient  symbolic  generation  methods  reported  in  mm-  We  out¬ 
line  a  procedure  for  constructing  polynomials  p  such  that  £/(p)  =  G(p),  where 
G  £  R[A'],  from  a  list  of  automatically  generated  Darboux  polynomials  (up  to 
some  given  degree).  The  procedure  will  require  two  lemmas  given  below. 

We  note  first  that  whenever  q  is  a  Darboux  polynomial,  any  constant  multiple 
of  q ,  i.e.  aq  for  some  a  £  R  or  C,  is  also  Darboux.  A  similar  property  holds  for 
the  decoupling  abstract  basis  functions  in  simulating  abstractions. 

Lemma  1.  If  p  £  M[Ai, . . . ,  Xn]  is  such  that  £f(p)  =  G(p)  where  G  £  R[A'], 
then  s  =  ap  +  b  for  any  real  numbers  a,  b,  is  such  that  £f(s)  =  F(s),  where 
FeR[l], 

Proof.  If  a  =  0  then  £/(s)  =  11/(6)  =  0  and  F  is  simply  the  zero  polynomial  in 
R[A].  If  a  ^  0,  by  our  hypothesis  we  have  £/(p)  =  G(p).  Let  us  write  p  = 
and  note  that 

£/(s)  =  Zf{ap  +  b)  =  a£,f(p )  +  £/(&)  =  a£/(p)  =  aG(p )  =  aG 

We  see  that  £/(s)  =  aG  ( s~“)  is  a  polynomial  in  s  with  real  coefficients.  □ 

One  consequence  of  Lem. [I]  is  that  whenever  we  assume  the  existence  of  a 
polynomial  p  such  that  £/(p)  =  G(p)  for  some  G  £  R[A],  it  always  suffices  to 
assume  the  existence  of  a  decoupling  abstract  basis  polynomial  p  —  r  for  any  real 
number  r. 

In  Prop.[2]we  established  that  the  existence  of  decoupling  abstract  basis  poly¬ 
nomials  p  is  related  to  the  existence  of  a  special  Darboux  polynomial  p  —  c  for 
some  complex  number  c.  For  any  polynomial  s,  we  denote  by  s*  the  polynomial 
obtained  by  setting  the  constant  term  of  s  to  zero.  For  instance,  if  s  =  x  +  1 
then,  s*  =  x.  Thus,  for  the  (Darboux)  polynomial  p  —  c,  one  has  ( p  —  c)*  =  p* 
(by  definition  of  the  *  operator)  and  therefore  p*  is  a  decoupling  abstract  basis 
polynomial  by  Lem.[l]  since  it  is  an  offset  of  the  polynomial  p  by  a  real  num¬ 
ber  (the  constant  term  of  p).  Therefore,  if  one  generates  Darboux  polynomials 
over  the  complex  numbers  and  finds  a  Darboux  polynomial  q  such  that  q*  is  a 
polynomial  over  the  reals  (i.e.  all  the  coefficients  of  q*  are  real  numbers),  then 
q*  is  potentially  a  decoupling  abstract  basis  polynomial,  which  can  be  checked 
by  solving  a  linear  program,  i.e.  by  running  REWHiTE(£/(g*),  g*),  as  outlined 
in  SectionEPl 
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Nevertheless,  generating  Darboux  polynomials  over  the  complex  numbers  will 
not  necessarily  return  Darboux  polynomials  q  such  that  q*  is  a  polynomial  over 
the  reals  even  if  the  latter  exist.  For  instance,  if  q  =  x2  +  xy  +  c  is  a  Darboux 
polynomial  with  some  complex  constant  term  c,  then  the  procedure  may  return 
iq  instead  of  q  (i  being  the  imaginary  number  satisfying  i2  =  —1),  although 
we  are  rather  interested  in  looking  for  q.  Enforcing  such  a  constraint  in  the 
procedure  for  generating  Darboux  polynomials  will  require  solving  mixed  non¬ 
linear  equations  where  some  variables  are  real  and  some  are  complex  numbers.  To 
avoid  solving  mixed  problems,  we  can  easily  adapt  the  generation  procedure  to 
produce  monic  Darboux  polynomials  for  any  variable  ordering,  for  instance  the 
lexicographic  order  X\  >  •  •  •  >  Xn.  Recall  that  monic  univariate  polynomials  are 
those  polynomials  where  the  leading  coefficient  (i.e.  the  coefficient  of  the  leading 
monomial)  is  equal  to  1.  In  the  multivariate  case,  the  notion  of  leading  coefficient 
additionally  requires  a  monomial  ordering.  For  instance,  for  the  order  X\  >  X2 , 
the  leading  monomial  of  the  polynomial  2X1X2  +  Xf  is  X2  and  therefore  the 
leading  coefficient  is  1 ,  whereas  the  leading  monomial  in  the  reverse  lexicographic 
ordering  X2  >  X\  is  XiX2  and  the  leading  coefficient  is  2. 

Lemma  2.  Given  a  polynomial  q  £  C[Ad, . . .  ,Xn],  let  p  £  C[X1; . . .  ,Xn]  be  the 
monic  polynomial  LC|gy ,  where  LC (q)  is  the  leading  coefficient  of  q  with  respect  to 
some  fixed,  monomial  ordering.  There  exists  a  non-zero  complex  number  z  such 
that  ( zq )*  £  R[Ai, . . . , Xn]  if  and  only  if  p*  £  R[Ai,  . . . , Xn\ . 

Proof.  Suppose  there  exists  such  a  non-zero  complex  number  z  such  that  (zq)*  £ 
K[X!, .  ..,Xn\.  Since  zLC(q)  =  LC  (zq)  we  have  that  =  ^^y  =  ^y  =  p, 

therefore  LC^g)  (zq)  =  P  and  Lc(~q)  (zl)*  =  P* ■  Since  LC(,zg)  £  R,  we  have 
p*  £  R[Xl,  . . . ,  Xn\.  Conversely,  if  p*  £  R[A'i, . . . ,  Xn],  take  z  =  j-^^y  so  that 
(zq)*=p*.  '  □ 

We  now  describe  a  procedure  for  generating  decoupling  abstract  basis  poly¬ 
nomials.  Suppose  we  are  given  all  the  independent  Darboux  polynomials  in 
C[Xi, . . .  ,Xn ]  for  the  system  x  =  f(x)  up  to  some  degree  d  >  0.  By  Prop.[2j 
if  there  exists  a  polynomial  p  £  R[A'i, . . . ,  Xn\  of  degree  d!  <  d  such  that 
=  G(p),  where  G  £  R[A]  is  non-constant,  then  there  necessarily  exists 
a  Darboux  polynomial  q  of  degree  d'  such  that  q*  is  a  polynomial  over  the  reals, 

i.e.  q*  £  R[Ad, . . . ,  Xn],  This  fact  suggests  a  simple  search  method.  Below  we 
describe  the  three  main  steps  in  the  procedure. 

1.  For  a  fixed  positive  integer  d,  automatically  generate  all  monic  Darboux 
polynomials  for  the  system  up  to  degree  d  with  coefficients  in  C. 

2.  For  each  generated  Darboux  polynomial  q  check  if  q*  £  R[A'i, . . . ,  Xn\  and 
if  so,  store  q*  as  a  candidate  in  a  list  L. 

3.  For  all  polynomials  q*  in  L,  run  Rewrite (£f(q*),q*).  If  q*  is  a  decoupling 
abstract  basis  polynomial,  the  rewriting  procedure  will  return  G  £  R[A]  s.t. 
£f(q*)  =  G(q*). 
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Example  4 ■  Consider  the  following  system 

X\  =  —(1  +  x\  —  2x2  +  2(1  +  (—  1  +  X\  +  #2)2)) 
o 

±2  =  ^(  — Xl  +  2a;2  +  (“I  +  X\  +  X2 )2) 

The  automatic  generation  procedure  for  Darboux  polynomials  over  C  up  to 
degree  1  gives  us  (<?i,  92,93)  =  (1+Xi  — 2x2,  (— 1-H)+Xi+x2,  (— 1— *)+Xi+x2).  In 
this  case,  ql,q2  and  are  all  candidates  for  the  short  list  L.  Since  q2  =  q$,  L  = 
{x\  —  2x2,x\  +  x2}.  Running  Rewrite(£/(<7*),  ql)  and  Rewrite(£/((72),  q2) 
returns  2  —  2X  +  X2  and  1  +  X,  respectively.  Thus,  letting  (ai,a2)  =  ( q*,q2 ), 
we  obtain  the  decoupled  abstraction: 

Ot\  —2  —  2 Gi\  d- 

d2  —  IT  a2. 

In  general,  a  Darboux  polynomial  q ,  with  q*  £  R[Xi, . . .  ,Xn],  is  not  nec¬ 
essarily  a  decoupling  abstract  basis  polynomial.  For  instance,  in  the  system 
Xi  =  XiX2,x2  =  x2,  one  has  X\  as  a  Darboux  polynomial;  however  x\  is  not  a 
decoupling  abstract  basis  polynomial  because  £/(xi)  =  X\X2  cannot  be  rewrit¬ 
ten  as  polynomial  in  only.  The  checking  procedure  Rewrite(£/(xi),  xi)  will 
thus  fail  to  produce  a  solution. 

It  is  natural  to  ask  under  what  extra  conditions  is  a  Darboux  polynomial  q 
satisfying  q*  £  R[X1; . . . ,  Xn]  also  a  decoupling  abstract  basis  polynomial.  The 
following  theorem  explores  this  connection. 

Theorem  1.  Given  a  system  of  polynomial  ODEs  x  =  f{x),  there  exists  a 
polynomial  p  £  R[Xi, . . . ,  Xn\  such  that  £f(p)  =  G(jp),  where  G  £  R[X] 
is  of  degree  d  >  0,  if  and  only  if  the  system  has  d  Darboux  polynomials 
q1,...,qd£C[X1,...,  Xn]  satisfying: 

(i)  q*  =  q2  =  ■  ■  ■  =  o*d  e  •  •  •  > xn\, 

(ii)  £f(qi)  =  &f(q2)  =  ■■■  =  £/(gd)  =  rqxq2  ■  ■  ■  qd,  r  £  R, 

(iii)  for  all  i  =  1, . . . ,  d,  either  q*  —  qi  £  R  or  there  exists  j  ^  i,  j  =  1, . . . ,  d, 
such  that  qi  =  qj . 

Proof.  Suppose  there  exists  a  p  £  R[Xl5 . . . ,  Xn]  such  that  £/(p)  =  G(p).  When 
G  £  R[X]  is  a  non-constant  polynomial  of  degree  d,  it  can  be  factorized  as 
r(X  —  c  1)  •  •  •  ( X  —  cd ),  where  and  the  roots  Ci  are  either  real  numbers,  or 

complex  numbers  that  come  in  conjugate  pairs,  i.e.  if  c;  £  C  is  a  root  of  G,  then 
its  complex  conjugate  Ci  is  also  a  root.  In  the  proof  of  Prop. [2]  we  have  seen  that 
for  any  such  factor  ( X  —  cf)  the  polynomial  qi  =  p  —  Ci  is  a  Darboux  polynomial 
for  the  system  such  that  £/(<7i)  =  G(p).  The  properties  (i),  (ii)  and  (iii)  follow 
immediately. 

Conversely,  let  us  assume  that  there  are  d  Darboux  polynomials  qi ,  q2 , . . . ,  qd 
satisfying  properties  (i),  (ii)  and  (iii).  Then  for  any  r  £  R  we  have 

rqiq2  ■  ■  ■  qd  =  r(q\  -  ci)^  -  c2)  ■  ■  ■  (qd  -  cd), 
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where  each  a  =  q*  —  qt  is,  by  definition,  a  constant.  By  property  (i)  we  have 
Qi  =  ?2  =  ‘ ' '  =  e  R[^i,  •  ■  ■ ,  Xn\,  so  let  us  take  p  =  g*  =  q2  =  ' '  ’  =  Qd 
obtain  r(ql-c1)(q%-c2)  ■  ■  ■  (q*d-cd)  =  r(p-c1){p-c2)  ■  ■  ■  ( P~cd ).  One  can  now 
write  this  as  r(p  —  c±)(p  —  c2)  ■  ■  ■  (p  —  cd)  =  G(p),  where  G  £  R[X]  has  degree 
d.  The  coefficients  of  G  are  real  because  by  (iii)  the  roots  Cj  come  in  complex 
conjugate  pairs.  Since  qt  =  q*  —  (5*  —  qi)  =  p  —  Ci,  we  have  £f(qt)  =  £f(jp  —  Ci)  = 
£f(p)  -  £/(<*)  =  £f(p)  and  by  (ii)  £f(p)  =  r(p  -  a ){p-c2)  ■  ■  ■  (p-  cd)  =  G{p). 

□ 

Notice  that  Rewrite  does  not  require  all  of  the  d  Darboux  polynomials 
in  order  to  construct  G.  If  a  family  of  Darboux  polynomials  {qi,...,qd}  as 
stated  in  Theorem[l]  exists,  it  suffices  to  supply  only  one  element,  say  q\,  to 
Rewrite,  which  will  then  find  a  rewriting  of  £f(q*)  as  G{q\)1  with  G  £  R[X]. 
If  however,  the  algorithm  fails,  then  the  polynomial  supplied  was  not  obtained 
from  such  a  family  of  Darboux  polynomials  and  therefore  cannot  be  used  to 
obtain  a  rewriting  of  its  derivative  in  terms  of  itself. 

Theorem[l]  exposes  the  structure  inherent  in  systems  for  which  one  can  find 
decoupled  simulating  abstractions.  The  requirements  (i)-(iii)  are  indeed  quite 
strong.  Observe  that  when  G  is  a  linear  polynomial  with  a  real  coefficient  A,  i.e. 
is  of  the  form  G( X)  =  AX  and  therefore  necessarily  has  one  real  root,  Theorem[l] 
reduces  to  the  conditions  for  constant-scale  consecution  [25], 

Remark  4-  Theorem[l]relies  on  generating  Darboux  polynomials  in  order  to  com¬ 
pute  a  decoupling  abstraction  of  a  given  system  of  polynomial  ODEs.  Nev¬ 
ertheless,  polynomials  having  constant  Lie  derivatives  (that  is,  those  p  s.t. 
£f(p)  =  G(p)  where  G  has  degree  zero)  can  also  be  used  for  decoupling  abstrac¬ 
tions,  but  are  not  covered  by  Theorem|I]  which  requires  the  degree  of  G  to  be  pos¬ 
itive.  The  special  case  when  G  has  degree  zero  is  also  related  to  Darboux  polyno¬ 
mials  as  follows:  (i)  when  G  is  the  zero  polynomial,  then  the  system  has  a  first  in¬ 
tegral  which  is  a  special  Darboux  polynomial  as  discussed  in  Section[2j  (ii)  when 
G  is  a  non-zero  constant,  then  the  augmented  system  (x,  t)  =  1)  obtained 

by  appending  the  time  derivative  to  the  original  system  has  a  polynomial  first  in¬ 
tegral.  More  precisely,  when  p  £  R[Xl,  . . . ,  Xn\  and  the  £f(p)  is  a  real  constant, 
say  r,  then  in  the  augmented  system  £(f,i) (p—rt)  =  £(/,i)(p)  —  r  =  £f(p)—r  =  0 
and  p  —  rt  is  thus  a  polynomial  first  integral  of  the  augmented  system.  One  may 
thus  handle  this  case  by  computing  first  integrals  (e.g.  using  the  approach  de¬ 
scribed  in  mi)  before  searching  for  more  sophisticated  decoupling  polynomials 
where  G  has  a  positive  degree. 

4  Outlook 

Verification  problems  for  systems  of  ODEs  can  be  soundly  translated  to  verifi¬ 
cation  problems  for  their  simulating  abstractions.  Below  we  sketch  the  case  of  a 
standard  safety  verification  problem  (SXl  /,  Fx),  where  one  wishes  to  prove  that 
a  given  property,  encoded  as  the  region  Fx  C  Rra ,  is  always  satisfied  if  the  system 
x  =  f(x)  is  initialised  in  Xq  £  Sx  C  R".  If  a  decoupling  abstraction  a  =  G(a) 
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exists,  one  can  attempt  to  solve  the  simpler  abstract  safety  verification  prob¬ 
lem  (Sy,  G,  Fy )  where  (y i, . . . ,  ym)  =  (ai(x), . . . ,  am( x)),  denoted  henceforth  by 
y  =  a(x),  i.e.  y  =  G(y)  is  a  decoupled  simulating  abstraction.  The  initial  set 
in  the  new  abstract  coordinates,  Sy  C  Rm  (resp.  Fy),  is  computed  as  a  projec¬ 
tion  of  the  semialgebraic  set  Sx  Ay  =  ot(x),  which  is  a  subset  of  R"+m  (resp. 
Fx  Ay  =  ot(x)),  onto  Rm.  Such  a  projection  can  in  principle  be  obtained  by 
eliminating  the  existential  quantifiers  in  the  following  sentence 

3  (xi , . . . ,  xn )  £  R  .  Sx  A  yi  —  ot i (xi , . . . ,  Xn)  A  •  A  ym  —  {x\ , . . . ,  xn^j . 

The  soundness  of  such  an  abstraction  relies  essentially  on  two  facts:  (i)  the  sets 
Sy  and  Fy  are  the  exact  images  through  a  of  the  sets  Sx  and  Fx  respectively 
(although  using  over-approximations  of  these  sets  is  also  sound)  and  (ii)  the 
invariant  regions  of  the  decoupled  abstract  system,  when  expressed  in  terms 
of  the  old  coordinates,  define  invariant  regions  of  the  original  system  (i.e.  the 
abstraction  is  indeed  sound  [22J  Theorem  2.2]).  This  means  that  if  the  safety 
problem  holds  true  in  the  decoupled  abstraction  it  also  holds  true  in  the  original 
concrete  system.  If  not,  however,  the  abstraction  may  be  too  coarse. 

Interesting  directions  for  refining  the  abstraction  include  searching  for  more 
general  simulating  abstractions  that  are  not  necessarily  completely  decoupling. 
For  instance,  it  is  conceivable  that  a  simulating  abstraction  may  possess  inde¬ 
pendent  sub-systems  that  are  of  the  form 

oi-i  Gi(ai,  ctj), 

Cij  —  Gj(<Xi,  Ctj ) , 

where  Gj,  Gj  £  R[X1;  X2]  and  a*,  ctj  £  R[X1; . . . ,  Xn]  are  the  abstract  basis  func¬ 
tions.  This  idea  is  similar  to  the  so-called  algebraizing  transformations,  briefly 
discussed  in  [25],  Definition  2.4],  The  analysis  of  2-dimensional  (i.e.  planar)  poly¬ 
nomial  ODEs  is  however  vastly  more  difficult  than  the  1-dimensional  case.  In¬ 
deed,  qualitative  analysis  of  planar  polynomial  flows  is  an  active  area  of  math¬ 
ematical  research  (e.g.  see  [615]  I.  However,  one  hope  is  this  greater  generality 
would  make  simulating  abstractions  of  this  form  more  “common”  in  systems 
that  one  might  encounter  in  applications. 

Decoupling  can  help  overcome  some  of  the  scalability  issues  in  existing  ver¬ 
ification  methodologies.  For  instance,  in  reachability  analysis,  relational  ab¬ 
straction  [24]  seeks  to  abstract  the  flow  of  a  differential  equation  by  an  over¬ 
approximation  of  the  reachability  relation  on  the  states  of  the  system.  Mathemat¬ 
ically,  a  (timeless)  relational  abstraction  of  an  autonomous  system  x  =  /( x)  is  a 
relation  R  C  Rra  x  R"  such  that  (x,y)  £  R  if  y  is  reachable  from  x  in  finite  time 
by  following  the  flow  of  the  system  [24j  Definition  4],  i.e.  if  3t  >  0.  tpt{x)  =  y. 
Computing  timeless  relational  abstractions  for  non-linear  systems  is  difficult  be¬ 
cause  it  reduces  to  searching  for  positive  invariants  in  the  extended  system  of 
ODEs  y  =  f(y),x  =  0  with  dimension  2 n,  i.e.  with  twice  the  number  of  state 
variables  J2U  Definition  5,  Lemma  1].  When  the  system  is  uncoupled,  one  can 
instead  work  with  n  extended  systems  yr  =  fi(yi),Xi  =  0,  i  =  1, . . . , n,  each  of 
dimension  2. 
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5  Related  Work 


Our  work  is  closest  in  spirit  to  that  of  Sankaranarayanan  |22] .  which  studied 
simulating  abstractions  resulting  from  linearizing  change  of  basis  transforma¬ 
tions.  Our  approach  instead  focused  on  simulating  abstractions  obtained  via 
decoupling  change  of  basis  transformations. 

Change  of  basis  transformations  are  a  standard  technique  for  decoupling  lin¬ 
ear  homogeneous  systems  of  ODEs  with  constant  coefficients,  i.e.  systems  of  the 
form  x  =  Ax,  where  A  is  an  n  x  n  real  matrix.  A  common  technique  applies  when 
the  matrix  A  has  n  real  distinct  eigenvalues  and  produces  a  decoupled  linear  ho¬ 
mogeneous  system  a  =  Bey  of  the  same  dimension ,  where  a  =  (oq, . . .  ,an)  is 
made  up  of  linear  functions  a.i  :  R"  — >  R  in  the  state  variables  x±, ...  ,xn  (see 
e.g.  EH  §28.2,  §28.3]);  in  particular,  such  a  decoupling  is  always  possible  when 
A  is  a  real  symmetric  matrix.  In  our  work,  we  consider  more  general  polynomial 
systems  of  ODEs  and  a  more  general  class  of  polynomials  to  act  as  the  new  basis; 
additionally,  we  do  not  require  the  dimension  of  the  resulting  decoupled  system 
to  match  that  of  the  original  system  of  coupled  ODEs.  In  short,  our  focus  is  not 
placed  on  solving  the  system,  but  rather  on  automatically  discovering  simulating 
abstractions  that  are  more  amenable  to  analysis. 

Girard  and  Pappas  explored  approximate  bisimulation  of  continuous  systems 
in  ]5],  and  Pappas  earlier  developed  (exact)  bisimulations  between  continuous 
linear  systems  [IS]-  However,  these  works  employ  a  different  notion  of  simulation 
and  do  not  seek  to  make  the  structure  of  the  simulation  easier  to  analyze  in  the 
way  that  we  do  with  decoupling,  and  are  in  practice  limited  to  linear  ODEs  due 
to  reliance  on  solving  linear  matrix  inequalities  (LMIs).  Han  and  Krogh  have 
also  explored  sound  order  reduction  techniques  for  verification  with  reachability 
analysis,  but  their  approach  is  also  limited  to  linear  ODEs  [12] .  In  contrast  to  all 
these  existing  works  that  employ  different  techniques  as  well  as  different  formal 
development,  our  decoupled  simulating  abstractions  are  applicable  to  non-linear 
polynomial  ODEs,  and  as  such,  are  developed  using  significantly  different  meth¬ 
ods. 

6  Conclusion 

In  this  paper  we  explored  a  technique  for  constructing  decoupling  simulating 
abstractions  of  non-linear  polynomial  ODEs,  which  can  be  more  easily  analyzed 
because  their  1-dinrensional  sub-systems  may  be  treated  independently.  We  em¬ 
ployed  the  theory  of  Darboux  polynomials  to  give  a  sufficient  criterion  for  non¬ 
existence  of  decoupled  simulating  abstractions  (up  to  a  some  maximum  degree  of 
the  abstract  basis  polynomials;  see  Prop. [2]).  Lastly,  we  described  how  automati¬ 
cally  generated  Darboux  polynomials  (up  to  some  given  polynomial  degree)  can 
be  used  to  construct  abstract  basis  polynomials  that  can  yield  decoupling  sim¬ 
ulating  abstractions.  The  abstractions  developed  in  this  paper  are  in  essence  a 
form  of  model  transformation,  which  can  be  integrated  in  source  transformation 
and  translation  tools  such  as  HyST  [2] ;  we  leave  this  for  future  work. 
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ABSTRACT 

This  paper  studies  ways  of  constructing  meaningful  opera¬ 
tional  models  of  piecewise-smooth  systems  (PWS).  The  sys¬ 
tems  we  consider  are  described  by  polynomial  vector  fields 
defined  on  non-overlapping  semi-algebraic  sets,  which  form  a 
partition  of  the  state  space.  Our  approach  is  to  give  meaning 
to  motion  in  systems  of  this  type  by  automatically  synthe¬ 
sizing  operational  models  in  the  form  of  hybrid  automata 
(HA).  Despite  appearances,  it  is  in  practice  often  difficult 
to  arrive  at  satisfactory  HA  models  of  PWS.  The  different 
ways  of  building  operational  models  that  we  explore  in  our 
approach  can  be  thought  of  as  defining  different  semantics 
for  the  underlying  PWS.  These  differences  have  a  number  of 
interesting  nuances  related  to  phenomena  such  as  chatter¬ 
ing,  non-determinism,  so-called  mythical  modes  and  sliding 
behaviour. 

Keywords 

piecewise-smooth  systems,  hybrid  automata,  operational 
models,  discontinuous  differential  equations. 

1.  INTRODUCTION 

Many  processes  in  which  smooth  continuous  motion  can 
be  interrupted  by  discrete  events  can  be  represented  by  or¬ 
dinary  differential  equations  (ODEs)  with  discontinuities. 
As  such,  they  are  part  of  a  broader  class  of  dynamical  sys¬ 
tems,  known  as  hybrid  (also  cyber-physical)  systems,  which 
combine  discrete  and  continuous  behaviour  under  a  unified 
frameworljj]  Hybrid  systems  are  increasingly  used  in  mod¬ 

*This  work  was  supported  by  the  Air  Force  Research  Labo¬ 
ratory  (AFRL)  through  contract  number  FA8750-15-1-0105 
and  the  Air  Force  Office  of  Scientific  Research  (AFOSR) 
under  contract  numbers  FA9550-15-1-0258  and  FA9550-16- 
1-0246. 

1Indeed,  some  of  the  earliest  research  in  hybrid  systems,  e.g. 
in  the  work  of  Witsenhausen  [44] ,  began  by  considering  pre¬ 
cisely  the  systems  where  there  are  no  “jumps”  in  the  contin¬ 
uous  state,  but  abrupt  changes  in  the  dynamics  are  possible. 
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elling  and  analyzing  the  behaviour  of  modern  control  sys¬ 
tems  employing  embedded  devices. 

Systems  described  by  discontinuous  ODEs  are  sometimes 
referred  to  as  piecewise-smooth  systems  (PWS).  Their  repre¬ 
sentation  has  proved  popular  in  the  control  systems  commu¬ 
nity  because  it  provides  a  concise  and  convenient  notation. 
However,  a  discontinuous  system  of  ODEs  explicitly  only 
conveys  information  about  the  continuous  dynamics  of  the 
system,  along  with  a  set  of  regions  where  state  evolution 
is  smooth;  the  discrete  transition  behaviour  of  the  system 
between  these  regions  is  not  explicitly  elaborated. 

There  exist  a  number  of  specification  formalisms,  such  as 
hybrid  automata  lj  and  hybrid  programs  [34],  whose  seman¬ 
tics  is  clearly  defined  and  which  can  serve  as  operational 
models  for  hybrid  systems.  Hybrid  automata  in  particular 
have  become  very  popular  in  the  verification  community.  In 
a  hybrid  automaton,  the  discrete  transition  behaviour  of  the 
hybrid  system  is  specified  explicitly,  which  can  often  make 
these  automata  large  and  unwieldy  even  when  specifying  hy¬ 
brid  systems  of  relatively  modest  size.  As  a  specification  for¬ 
malism,  discontinuous  ODEs  provide  a  much  more  concise 
and  manageable  description  of  piecewise-smooth  systems,  al¬ 
beit  leaving  many  important  details  about  their  behaviour 
implicit. 

Researchers  working  in  computer  science  and  control  sys¬ 
tems  tend  to  put  different  emphasis  on  the  importance  of  for¬ 
mal  modelling  and  tend  to  use  significantly  different  meth¬ 
ods  to  model  and  reason  about  systems.  One  particular 
aspect  of  these  differences  is  manifest  in  the  temptation 
to  treat  hybrid  automata  naively  as  being  merely  syntactic 
variants  of  discontinuous  ODEs  when  modelling  piecewise- 
smooth  systems.  Subscribing  to  this  view  is,  however,  rather 
dangerous  and  can  lead  to  unintended  behaviour  in  the  re¬ 
sulting  models. 

In  this  paper  we  study  the  challenges  presented  by  the 
problem  of  transforming  concise  descriptions  of  piecewise- 
smooth  systems  in  the  form  of  discontinuous  ODEs  into 
formal  operational  models  in  the  form  of  hybrid  automata. 
Transformations  that  result  in  satisfactory  models  are,  as 
we  shall  see,  far  from  trivial  to  both  formulate  and  ef¬ 
fect.  We  develop  automatic  procedures  for  transforming 
piecewise-smooth  systems  with  polynomial  dynamics  and 
semi-algebraic  constraints  into  hybrid  automata. 

A  number  of  different  interpretations  of  the  operational 
meaning  of  piecewise-smooth  systems  are  possible,  creating 
a  degree  of  ambiguity  about  their  intended  behaviour  (i.e. 
their  semantics);  this  gives  rise  to  significant  differences  in 
the  form  and  the  behaviour  of  the  hybrid  automata  that  one 
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can  construct.  A  number  of  important  choices  can  be  exer¬ 
cised  in  transforming  PWS  to  HA  in  order  to  ensure  that 
the  resulting  operational  models  reflect  the  desired  interpre¬ 
tation. 

1.1  Contributions 

In  this  paper  we  describe  a  method  for  automatically  con¬ 
structing  hybrid  automata  from  descriptions  of  piecewise- 
smooth  polynomial  systems  and  thus  build  their  operational 
models.  We  discuss  aspects  of  the  semantics  of  transi¬ 
tions  directly  related  to  phenomena  such  as  chattering,  non¬ 
determinism  and  the  presence  of  so-called  mythical  modes  in 
the  underlying  systems.  We  illustrate  how  our  technique  can 
be  applied  to  model  systems  with  so-called  sliding  modes. 
We  conclude  with  a  discussion  of  related  work  and  an  out¬ 
look  for  future  research. 

2.  MATHEMATICAL  PRELIMINARIES 

2.1  Continuous  systems  and  vector  fields 

A  general  n-dimensional  autonomous  system  of  first-order 
ODEs  has  the  form: 

xi  =  fi(xi,x2,  ■  ■  ■  ,x„), 


in  =  fn(x  1,X2,  ■  ■  ■  ,Xn), 

where  /;  :  Rn  — >  R  are  real- valued  functions  (typically  C1) 
for  each  i  =  1  ,...,n  and  ii  denotes  the  derivative  of  Xi 
with  respect  to  time,  i.e.  -Tt.  Such  a  system  defines  a 
vector  field  f  :  Rn  — >  Rn,  where  f(x)  =  (/i(x), . . . ,  fn(x)) 
for  any  x  £  Rn.  We  will  denote  autonomous  systems  of 
ODEs  using  the  more  concise  vector  field  notation,  i.e.  by 
writing  x  =  f(x). 

In  applications,  it  is  often  the  case  that  the  state  of  the 
system  is  required  to  only  evolve  within  some  prescribed 
set  of  “legal”  states  M  C  Rn,  which  is  known  as  the  mode 
invariant,  or  evolution  constraint.  We  will  express  this  re¬ 
quirement  concisely  by  writing  x  =  f(x),  x  £  M.  When  no 
evolution  constraint  is  specified,  M  is  assumed  to  be  R“. 

A  solution  to  the  initial  value  problem  for  the  system  of 
ODEs  x  =  f(x)  with  initial  value  xg  £  Rn  is  a  differen¬ 
tiable  function  x  :  (a,  b)  — >  R™,  where  x(f)  is  defined  for  all 
t  within  some  non-empty  extended  real  interval  including 
zero,  i.e.  t  £  (a,  b)  CRU  {oo,  — oo}  where  a  <  0  <  b,  and 
such  that  x(0)  =  xo  and  yjx(t)  =  f(x(f))  for  all  t  £  (a,b). 
In  what  follows,  we  will  denote  the  solution  x(t)  by  writ¬ 
ing  <fit(x o),  to  emphasize  the  initial  value.  If  the  function 
o)  is  available  in  closed-fornj^]  one  can  analyze  the  tem¬ 
poral  behaviour  of  the  system  initialized  in  the  state  xo  by 
analyzing  the  closed-form  expression.  In  practice,  however, 
it  has  long  been  established  that  explicit  closed-form  solu¬ 
tions  to  non-linear  ODEs  are  highly  uncommon  [20  . 

Systems  of  ODEs  whose  right-hand  sides  are  locally 
Lipschitz  continuous  (e.g.  polynomial  functions  fall  un¬ 
der  this  class)  guarantee  existence  of  unique  solutions  on 
some  non-trivial  time  interval  (a,  b)  for  any  initial  value 
xo  £  Rn  (by  the  Cauchy-Lipschitz/Picard-Lindelof  theorem; 
see  e.g.  [39]). 

2  By  this  we  understand  a  finite  expression  in  terms 
of  polynomials  and  elementary  special  functions  such  as 
sin,  cos,  exp,  In,  etc. 


2.2  Piecewise-smooth  vector  fields 

Given  a  partition  of  some  set  M  C  Rn  into  finitely  many 
non-overlapping  subsets  M\ , ... ,  Mm ,  we  consider  a  finite 
family  of  vector  fields  fi  :  Rn  — >  R™,  where  i  £  {1, . . . ,  m}. 
By  assigning  the  vector  field  fi  from  this  family  to  the  set  Mi 
for  each  i  =  1, . . . ,  m,  we  arrive  at  a  vector  field  5  :  M  — ¥  R™ 
which  is  defined  piecewise  on  M,  i.e. 


fl(x) 

x  £  Mi , 

fm(x) 

X  £  Mm 

At  this  point,  let  us  remark  that  while  the  sets  Mi, . . . ,  Mm 
need  not  be  differentiable  manifolds,  the  corresponding  vec¬ 
tor  fields  fi , . . . ,  f m  are  defined  on  Rn .  It  is  therefore  mean¬ 
ingful  to  speak  about  motion  occurring  within  the  manifold 
Rn  according  to  the  systems  of  ODEs  x  =  fi(x),  but  con¬ 
fined  to  the  states  within  Mi.  With  this  intuition,  the  vector 
field  5  can  be  interpreted  as  describing  a  system  of  ODEs 
x  =  5(x)  with  a  piecewise-defined  (and  potentially  discon¬ 
tinuous)  right-hand  side,  i.e.  explicitly  given  by 

x  =  3r(x)  (2) 

To  precisely  describe  the  motion  taking  place  (within  the 
set  M)  in  such  a  system,  in  general  one  may  no  longer  call 
upon  the  classical  notion  of  solution  developed  for  contin¬ 
uous  ODEsJ^J  Indeed,  there  is  no  single  universally  agreed- 
upon  definition  of  solution  for  systems  of  ODEs  with  dis¬ 
continuities.  Extensions  of  the  classical  notion,  such  as 
Caratheodory  solutions,  among  others  [19],  have  been  sug¬ 
gested,  but  these  differ  in  the  way  they  model  certain  dy¬ 
namic  behaviours  and  therefore  give  different  meaning  (i.e. 
semantics)  to  systems.  An  excellent  accessible  survey  of 
discontinuous  ODEs  and  the  various  generalized  solution 
concepts  developed  for  them  was  given  by  Cortes  in  1 8] . 
Intuitively,  one  expects  generalized  solutions  to  piecewise- 
smooth  systems  to  be  continuous  functions  of  time,  because 
these  systems  do  not  allow  for  discontinuous  jumps  in  their 
(continuous)  state,  but  with  the  differentiability  requirement 
for  the  solution  (in  some  way)  appropriately  relaxed.  Solu¬ 
tions  for  more  general  classes  of  hybrid  systems  (which  may 
allow  discontinuous  jumps  in  the  state)  are  trickier,  and  re¬ 
quire  generalized  time  domains,  such  as  hybrid  time  domains 
explored  in  the  work  of  Sanfelice,  Goebel  and  Teel  [36;  |l8|. 
In  our  approach,  we  will  not  directly  make  use  of  these  no¬ 
tions,  relying  instead  on  the  semantics  of  hybrid  automata 
(after  Lygeros  et  al.  [27]),  which  we  shall  describe  presently. 

2.3  Hybrid  automata  as  operational  models 

Hybrid  automata  were  first  introduced  by  Alur  et  al.  1 
as  a  formal  specification  language  for  hybrid  systems.  They 
provide  operational  models  for  hybrid  systems  in  the  same 
way  that  transition  systems  provide  models  for  discrete  com¬ 
puter  programs,  making  it  possible  to  give  a  precise  math¬ 
ematical  description  of  their  execution.  We  will  employ  the 
term  evolution  when  speaking  about  hybrid  systems  (just  as 
with  continuous  systems)  and  use  the  term  execution  only  in 
the  context  of  operational  models,  such  as  hybrid  automata. 

As  formal  models,  hybrid  automata  have  been  used  exten¬ 
sively  in  both  modelling  [12]  and  verification  of  properties 

3E.g.  it  is  continuity  of  the  right-hand  side  that  guarantees 
the  existence  of  solutions  (by  Peano’s  theorem). 
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in  hybrid  systems  [14|  |40] .  Below  we  reproduce  a  very  con¬ 
venient  definition  of  hybrid  automata  and  their  execution, 
due  to  Lygeros  et  al.  [27];  for  alternative  definitions  the  in¬ 
terested  reader  is  invited  to  consult  01221111- 

Definition  1.  Formally,  a  hybrid  automaton  HA  is 
given  by  an  8-tuple 

HA  =  (Q,  X,  F ,  Init,  Dorn,  E,  G,  R), 
where  the  elements  are  as  follows: 

•  Q  =  {qi,  q2,  ■  ■  ■ ,  qm}  is  a  finite  set  of  discrete  states, 

•  X  =  Rn  is  a  set  of  continuous  states, 

•  F  :  Q  x  X  — »  Rn  is  a  vector  field, 

•  Init  CQxX  is  a  set  of  initial  states, 

•  Dom  :  Q  — >  2X  is  a  mode  domain  (also  invariant,), 

•  E  C  QxQ  is  a  set  of  edges  (also  discrete  transitions,), 

•  G  :  E  — >  2X  is  a  guard  condition, 

•  R  :  E  x  X  — >  2X  is  a  reset  map. 


Standard  assumptions  with  this  definition  are  that  guard 
conditions  are  non-empty  whenever  they  are  specified,  i.e. 
for  all  e  £  E  it  is  the  case  that  G(e )  f  0  and  also  that  reset 
maps  can  only  take  the  system  to  a  genuine  continuous  state, 
i.e.  for  all  x  £  G(e),  R(e,x)  f  0. 


2.3.1  Semantics  of  hybrid  automata 

A  hybrid  time  trajectory  is  a  finite  or  infinite  sequence  of 
contiguous  time  intervals  starting  at  0,  where  the  end  points 
are  interpreted  as  times  at  which  a  discrete  event ,  such  as 
a  transition,  occurs.  More  formally,  following  [27],  a  hybrid 
time  trajectory  is  a  sequence  of  intervals  t  ==  ^ } JY_0 ,  for 
which  li  =  [ Ti,Ti\  for  all  i  <  N,  where  N  £  N  U  {oo}, 
and  n  <  f  =  n+\  for  all  i.  If  the  sequence  is  finite,  i.e. 
if  N  <  oo,  then  either  In  =  [tat,t^]  or  In  =  [tjv,t^). 
Intuitively,  one  may  think  of  n  as  the  times  at  which  discrete 
transitions  occur. 

An  execution  (or  a  run )  of  a  hybrid  automaton  is  defined 
to  be  the  triple  (r,q,ft{x)),  where  r  is  a  hybrid  time  tra¬ 
jectory,  q  :  (t)  — >  Q ,  where  (t)  is  defined  to  be  the  set 


{0, 1, . . . ,  IV}  if  r  is  finite  and  {0, 1, . . .  }  otherwise  27 


a  map  and  </?}( x)  is  a  collection  of  differentiable  functions 
ft(x)  '■  E  — >  R™  such  that  (q(0),  (Pq(x))  £  Init  and  for  all 
t  £  [n,  r'i)  it  is  the  case  that  x  =  F(q(i),  ip](x))  and  ipl(x)  £ 
Dom(g(j)).  It  is  also  required  that  transitions  respect  the 
guards  and  the  reset  maps,  i.e.  e  =  (q(i),q(i  +  1))  £  E, 
Tr'Xx)  £  G(e)  and  (ipl,(x),iplr+f(x))  £  R(e). 


3.  PROBLEM  OVERVIEW 

This  section  gives  an  overview  of  the  challenges  associated 
with  modelling  piecewise-smooth  systems  using  the  hybrid 
automaton  formalism. 

If  one  were  to  naively  translate  a  system  of  the  form  shown 
in  into  a  hybrid  automaton,  as  a  first  step  one  could 
simply  take  the  sets  Mi,...,Mm  to  be  the  mode  invari¬ 
ants  of  the  discrete  states  in  the  automaton  (i.e.  by  letting 
Dom  in  Definition  [I]  be  qi  K >  Mi  for  each  i  =  1,  ...,m) 
and  set  the  continuous  dynamics  within  these  modes  to  be 
governed  by  the  differential  equation  x  =  fi(x),  i.e.  let¬ 
ting  the  vector  field  F(qt,  x)  =  fi(x)  for  each  i  =  1, . . . ,  m, 
respectively.  The  resulting  hybrid  automaton  would  have 


|Q|  =  m  discrete  states  and  no  discrete  transitions  between 
them.  Clearly,  this  would  not  be  an  adequate  model,  since 
the  original  system  will  most  likely  evolve  into  and  out  of  the 
sets  Mi, . . . ,  Mm.  This  fact  raises  an  immediate  problem:  in 
order  to  have  discrete  transitions  in  the  hybrid  automaton 
one  is  required  to  specify  their  enabling  guards,  i.e.  sets 
of  states  within  the  mode  invariant  of  the  outgoing  discrete 
state  in  which  a  discrete  transition  is  possible. 


Figure  1:  Naive  construction  (mode  transitions  impossible). 


To  appreciate  the  problem  more  fully,  let  us  consider  a 
simple  1-dimensional  system  defined  on  the  partition  of  the 
real  line  R  into  three  regions:  x  <  0,  x  =  0  and  x  >  0,  and 
where  the  vector  fields  are  respectively  given  by  f i  (a;)  =  1, 
fa  (re)  =  2  and  f.i(x)  =  3  (i.e.  x  =  1,  x  =  2,  and  x  =  3) 
inside  each  region.  Clearly,  one  expects  this  system,  when 
started  inside  x  <  0,  to  transition  into  x  =  0  and  then 
to  x  >  0.  In  order  for  a  hybrid  automaton  to  faithfully 
model  the  behaviour  of  this  system,  we  require  two  discrete 
transitions  that  take  the  state  from  x  <  0  to  x  =  0  and 
from  x  =  0  to  x  >  0;  however,  in  the  former  transition  it  is 
not  possible  to  specify  x  =  0  to  be  the  guard  (as  shown  in 
Fig.  0  ,  since  this  set  lies  outside  of  the  mode  invariant  x  <  0 
of  the  outgoing  discrete  state.  It  is  possible  to  declare  the 
transition  guard  to  be  in  some  thin  layer  near  the  boundary, 
e.g.  8  <  x  <  0,  where  —  i  is  large,  but  any  such  choice  of  5 
would  be  rather  arbitrary  in  the  general  case.  Furthermore, 
there  would  remain  another  important  problem,  this  time 
with  the  latter  transition  from  x  =  0  to  x  >  0:  in  order  to 
make  such  a  transition  without  creating  discontinuities  (in 
this  case  “gaps”)  in  the  trajectory  through  reset  maps,  the 
state  of  the  system  needs  to  lie  within  the  mode  invariant 
of  the  destination  discrete  state  when  the  transition  guard 
is  enabled.  The  guard  x  =  0  is  thus  also  unsuitable  in  this 
case  and  there  is  no  easy  fix  to  this  problem. 

Instead  of  using  Mi , . . . ,  Mm  as  mode  invariants  in  the 
automaton,  one  may  instead  opt  to  use  their  closures 
Mi , . . . ,  Mm  with  a  view  to  enabling  the  transition  guards 
on  appropriate  subsets  of  the  boundaries  9Mi, . . . ,  dMm, 
which  would  now  lie  inside  the  corresponding  mode  invari¬ 
ants.  This  approach,  while  conceptually  simple,  has  a  num¬ 
ber  of  serious  deficiencies  and  results  in  hybrid  automata 
that  exhibit  chattering  runs ,  i.e.  can  perform  an  arbitrary 
number  of  discrete  transitions  without  advancing  the  con¬ 
tinuous  state  or  time. 

The  use  of  set  closures  additionally  overlooks  an  impor¬ 
tant  computational  drawback,  which  is  that  closures  are  typ¬ 
ically  very  difficult  to  compute  exactly  for  important  classes 
for  sets,  such  as  e.g.  semi-algebraic  sets  (i.e.  sets  described 
by  a  finite  Boolean  combination  of  polynomial  equalities  and 
inequalities;  see  e.g.  28  Definition  8.6.1]). 


Remark  1.  In  general  for  semi-algebraic  sets,  S  cannot 
be  obtained  from  S  by  syntactically  replacing  every  instance 
of  strict  inequalities  in  its  description  by  non-strict  inequal¬ 
ities  (e.g.  x3  —  x2  >  0  is  not  the  closure  of  x3  —  x2  >  0)  Q 
Remark  3.2].  The  closure  of  a  semi- algebraic  set  S  is  given 
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by  the  set 

S  =  {a;  £  Rn  |  Vr  >  0.  3 y  G  S.  || y  —  x\\2  <  r2}, 

where  the  norm  ||||  is  the  standard  Euclidean  distance  (see 
e.g.  Chapter  3]).  Let  the  set  S  be  described  by  a 
quantifier-free  formula  in  the  theory  of  real  arithmetic  with 
free  variables  xi , . . . ,  x„ .  By  performing  a  syntactic  substi¬ 
tution  of  the  free  variables  Xi  by  yi  (i  =  1, . . . ,  n)  everywhere 
in  the  formula,  one  obtains  a  quantifier-free  formula  in  the 
variables  yi, . . .  ,yn-  The  closure  S  can  then  be  characterized 
by  the  formula 

Vr  >  0.  3  yi, . . .  ,y„.  S  A  (yi  -  xi )2  H - 1 -  (yn  -  xn)2  <  r2, 

where  xi,...  ,xn  are  treated  as  fresh  free  variables  and  r  is 
a  fresh  bound  variable.  It  is  therefore  possible  to  apply  real 
quantifier  elimination  to  reduce  this  formula  to  an  equivalent 
one  that  is  quantifier-free  and  features  only  the  free  variables 

Xx ,  ■ * ■ j xn . 

Real  quantifier  elimination  (QE)  is  computationally  ex¬ 
pensive,  having  complexity  doubly- exponential  in  the  num¬ 
ber  of  quantifier  alternations  \l0 The  popular  CAD  algo¬ 
rithm  for  real  QE,  is  doubly- exponential  in  the  number  of 
variables  &  which  makes  it  impractical  for  problems  with 
a  large  number  of  variables  (using  currently  existing  imple¬ 
mentations). 

In  this  paper  we  pursue  a  very  different  approach  to  con¬ 
structing  HA  operational  models  of  PWS,  which  does  not 
require  computing  set  closures  ,  but  instead  requires  only 
the  “relevant”  subsets  on  their  boundaries  and  relies  funda¬ 
mentally  on  the  notion  of  “entry”  and  “exit”  sets  that  will  be 
the  subject  of  the  following  section. 

4.  OPERATIONAL  MODELS 

This  section  will  review  some  important  definitions  before 
presenting  an  algorithm  for  automatically  generating  HA 
operational  models  of  PWS. 

4.1  Fundamental  Definitions 

We  start  by  defining  an  important  set  that  will  shortly 
become  of  interest: 

Definition  2  (Inward  Crossing  Set). 

Enterj(S')  =  {x  G  Rn  |  3  e  >  0. 

V  t  G  (0,  e).  tpt(x)  G  S  A  V  t  G  (— e,  0).  <pt{x)  V  S} 

where  ift(x)  denotes  the  (unique)  solution  to  the  locally 
Lip schitz- continuous  system  of  ODEs  x  =  f(x). 

The  intuition,  as  suggested  by  the  name,  is  that  Enterj(5) 
describes  the  states  at  which  the  system  is  about  to  evolve 
inside  S,  after  having  only  just  evolved  outside  of  S.  Like¬ 
wise,  we  define  Exitf(S’)  to  be  the  set  of  states  at  which 
the  system  is  about  to  evolve  outside  of  S,  after  having 
only  just  evolved  inside,  i.e.  Exitf(S)  =  Enterf(-i.S'),  where 
-i S  :=  Rrl  \  S.  Note  that  such  states  need  not  necessarily  lie 
within  S  itself  and  may  lie  outside;  however,  they  necessar¬ 
ily  lie  on  the  boundary  of  S.  We  observe  that  the  crossing 
set,  by  its  very  definition,  can  be  expressed  by  means  of  one 
fundamental  building  block. 

Lemma  1  (Crossing  Set  Deconstruction). 
Enterf(S)  =  Inf(S')  D  In_f(-iS) ,  where 

In; (S')  =  {x  G  Rn  |  3  e  >  0.  V  t  G  (0,e).  <fit(x)  G  S}. 


Intuitively,  Inj(S)  denotes  the  states  in  Rn  from  which  the 
motion  of  the  system  takes  place  within  the  set  S  for  some 
time  segment  immediately  following  0  (i.e.  in  the  immediate 
future).  By  analogy,  when  considering  — f,  the  reverse  of  the 
vector  field  f,  In_f(S)  denotes  the  states  in  Rn  from  which 
the  motion  of  the  system  took  place  within  the  set  S  for 
some  time  segment  immediately  preceding  0  (i.e.  in  the 
immediate  past). 

In  the  special  case  when  the  system  x  =  f(*)  has  poly¬ 
nomial  right-hand  sides  and  S'  is  a  semi-algebraic  set,  the 
sets  Inf(S),  and  hence  In_f(S),  are  also  semi-algebraic  and 
can  be  computed  exactly  (a  result  due  to  Liu  et  al.  |25|). 
As  a  consequence,  the  sets  Enterj(S)  and  Exitf(S)  are  also 
computable  and  semi-algebraic  under  these  assumptions. 

We  stress  the  fact  that  the  boundary  of  S  need  not  be 
included  in  Enterf(S)  U  Exitf(S).  In  particular,  the  set 

Bouncej(S)  =  Inf(S)  fl  In_f(S) 

describes  those  states  that  may  leave  S  momentarily  at  a 
point  while  evolving  within  S  before  and  after  the  “bounce” 
and  can  therefore  lie  outside  of  Enterf(S)  U  Exitf(S).  Ap¬ 
pendix  [B]  provides  an  illustration  to  help  develop  some  in¬ 
tuition  about  the  meaning  of  these  sets. 

4.2  Generating  Hybrid  Automata 

We  now  have  at  our  disposal  the  machinery  necessary 
for  building  operational  models  of  piecewise-smooth  systems 
x  =  $(x),  i.e.  systems  of  the  form: 


fi(aO 

x  G  Mi  , 

f  m{x) 

x  G  Mm 

Given  such  a  system,  our  aim  is  to  synthesize  a  hybrid  au¬ 
tomaton  that  provides  an  adequate  model  of  the  behaviour 
of  the  system.  To  do  this,  our  approach  we  will  be  to  first 
augment  the  original  invariant  modes  of  the  system  Mi  with 
additional  states,  before  they  can  become  mode  invariants 
of  a  hybrid  automaton.  This  step  requires  a  definition. 

Definition  3.  Given  a  semi-algebraic  set  S  C  R™  and  a 
system  of  polynomial  ODEs  x  =  f(x),  the  augmented  set  of 
S  with  respect  to  this  system,  Aug(5,  f),  is  defined  by 

Aug(S,  f)  =  S  U  Enterf(S)  U  Exitf(S')  U  Bouncef(S) . 

In  the  context  of  piecewise-smooth  systems  of  the  form 
x  =  $(x),  whenever  we  wish  to  augment  the  set  Mi  with  re¬ 
spect  to  the  system  x  =  fi{x),  we  shall  adopt  a  more  concise 
notation  and  simply  write  9J L,  i.e.  ©L  =  Aug(M;,fj). 

The  definition  extends  each  invariant  mode  S  with  its  “en¬ 
try”,  “exit”  and  “bounce”  sets.  The  main  intuition  being  that 
if  the  system  was  to  enter  or  exit  the  mode  invariant  S  with 
respect  to  the  dynamics  f  then  it  will  do  so  by  necessarily 
crossing  those  sets.  The  set  Bouncef(S)  allows  the  evolution 
to  continue  within  S  after  “momentarily  exiting”  S.  If  in 
addition  the  mode  invariants  have  to  satisfy  a  global  con¬ 
straint  M,  then  it  should  be  accounted  for  by  intersecting  it 
with  Aug  (Mi,  fi). 

Algorithm  [I]  gives  a  pseudocode  procedure  for  generating 
a  hybrid  automaton  HAj.  The  procedure  begins  construct¬ 
ing  the  automaton  by  first  creating  m  distinct  discrete  states 
Q  =  {qx, . .  .  ,  gm}  (line  1),  defining  X  to  be  Rn  (line  2)  and 
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creating  a  set  of  edges  (i.e.  transitions)  E  by  computing  the 
Cartesian  product  Q  x  Q  and  removing  all  edges  of  the  form 
(qi,qi),  i.e.  removing  all  stuttering/self- looping  transitions 
(line  3).  It  then  proceeds  to  initially  assign  the  empty  set 
to  all  the  remaining  variables  on  line  4.  The  algorithm  then 
proceeds  to  create  the  modes  of  the  hybrid  automaton  HAj 
in  its  first  loop  (lines  5-12),  where  it  builds  the  extensional 
definition  of  F  by  assigning  the  vector  field  f;  to  the  discrete 
state  qi  (line  6),  augmenting  each  set  Mi  with  its  “entry”, 
“exit”  and  “bounce”  sets  (line  7)  and  using  this  to  build  an 
extensional  definition  of  the  Dom  mapping  (line  8)  which 
provides  the  mode  invariant  971;  for  each  state  qi  of  the  hy¬ 
brid  automaton.  Lines  9-11  are  responsible  for  converting 
the  initial  set  of  states  for  the  PWS  into  one  for  the  hy¬ 
brid  automaton  (this  step  can  in  fact  be  factored  out  of  the 
algorithm  and  performed  separately). 

The  second  loop  of  the  algorithm  (lines  13-16)  constructs 
the  discrete  transitions  and  is  responsible  for  defining  the 
discrete  transition  behaviour  of  the  resulting  automaton. 
The  loop  iterates  through  all  the  transitions  constructed  on 
line  3  and  defines  the  guards  (line  14)  and  reset  maps  (line 
15)  associated  with  each  transition.  The  reset  map  is  cho¬ 
sen  to  be  the  identity  and  therefore  does  not  affect  the  state 
of  the  system  upon  taking  any  transition.  Different  pos¬ 
sible  choices  for  the  guard  condition  GC (i,j)  (line  14)  are 
discussed  in  the  next  section. 


Data:  M  C  IT,  Mi, ,  Mm  C  M,  fi, . . . ,  fm  :  R"  -> 
Rn,X0  C  M 

Result:  Hybrid  automaton  HAj 

1  Q<r-  {qi,...,qmy, 

2  It-R"; 

3  E  <r-  Q  x  Q\{(qi,qi),  (92,92),...  (9m,  9m)}; 

4  Init,  Dom,  F,G,R<—  0; 

5  foreach  i  £  {1, . . . ,  m}  do 

6  I  {((qi,  x),  fi(as))}; 

7  9Jt;  <—  Aug(Mi,  fi)  n  M ; 

8  Dom  <—  Dom  U  {qi  9)1;}; 

9  if  A'o  PI  Mi  A  0  then 

10  |  Init  t—  Init  U  {(91,  x)  \  x  £  Mi  n  Ao} 

11  |  end 

12  end 

13  foreach  e  =  (qt,qj)  G  E  do 

14  G<-GU  {(e,  GC(i,  j))}; 

15  R  «-  RU  {((qi,qj),x  eA  {*})} 

16  end 

17  return  ( Q ,  X,  F,  Init,  Dom,  E,  G,  R) 

Algorithm  1:  Procedure  for  synthesizing  a  HA  from 
PWS. 


4.3  Discrete  Transition  Behaviour 

The  transition  guard  G(e)  =  GC {i,j),  i  A  j,  for  the  tran¬ 
sition  qt  — >  qj  entirely  determines  the  discrete  transition  be¬ 
haviour  of  the  automaton.  In  what  follows,  we  will  consider 
three  choices  for  this  formula. 

Remark  2.  We  stress  the  fact  that  these  are  by  no  means 
the  only  possible  semantics;  they  are  primarily  meant  to  ex¬ 
emplify  how  the  method  works  and  how  one  can  adapt  Al¬ 
gorithm  [7]  to  generate  operational  models  exhibiting  qualita¬ 
tively  different  behaviours. 


Recall  that  mode  qt  (resp.  qj)  has  mode  invariant  ©1,  (resp. 

I  =  A  OT;  A  lnfj  (Mj) 

II  =  I  A  -lEnterf,  (Mi)  A  -iBounce^.  (Mj) 

Ills  I  A  -Jnfi  (Mi) 

Informally,  these  formulas  are  characterizing  the  sets  of 
states  where  (1)  the  augmented  mode  invariants  9)1;  and 
dJtj  intersect  to  allow  for  continuous  transitions  and  (2) 
where  the  trajectory  of  the  system  in  mode  qj  can  evolve 
within  that  mode  for  some  time,  hence  the  intersection  with 
Inj .  (Mj).  Formulas  II  and  III  impose  additional  constraints 
on  the  guard.  Namely,  formula  II  additionally  requires  that 
the  guard  does  not  feature  states  in  the  intersection  of  the 
“entering”  set  of  the  outgoing  state  and  the  “bounce”  set  of 
the  incoming  state.  As  will  be  seen  in  later  sections,  this 
is  primarily  done  to  eliminate  so-called  chattering  in  the 
model.  Formula  III  is  different  in  that  it  only  enables  a 
transition  guard  if  no  further  continuous  motion  is  possible 
within  the  mode.  This  has  the  effect  that  transitions  must 
be  taken  precisely  when  they  are  enabled. 

Replacing  GC  (i,j)  in  line  14  of  Algorithm  [I]  by  formula  I, 
II  or  III  will  generally  result  in  a  different  operational  model 
which  can  exhibit  very  different  behaviour.  In  what  follows, 
we  will  refer  to  these  formulas  as  respectively  defining  guard 
conditions  of  type  I,  II  and  III. 


4.4  Computability 

An  operational  model  of  a  PWS  in  the  form  of  a  hy¬ 
brid  automaton  is  computable  using  Algorithm  [T]  when¬ 
ever  the  vector  fields  fi , . . . ,  fm  are  polynomial  and  the  sets 
Mi, . . . ,  Mm,  M  and  Xo  are  semi-algebraic. 

We  recall  that  a  set  is  semi-algebraic  if  it  is  char¬ 
acterized  by  a  finite  Boolean  combination  of  polyno¬ 
mial  equations  and  inequalities.  Thus,  the  formula 
xi  >  0  A  X2  =  0  V  x%  —  xi  <0,  where  the  symbols  xi,  X2  are 
interpreted  over  the  real  numbers,  characterizes  the  semi- 
algebraic  set  {(*1,2:2)  G  R2  |  *1  >  OA.X2  =  OV12-I1  <  0}. 
It  suffices  to  consider  formulas  without  quantifiers,  e.g.  V 
and  3,  since  the  theory  of  real  arithmetic  admits  quantifier 
elimination  [38]  and  therefore  any  formula  featuring  quanti¬ 
fiers  may  be  reduced  to  an  equivalent  quantifier-free  formula 
using  a  terminating  procedure^ 

It  was  shown  in  [25]  that  the  set  In^(5)  can  be  computed 
exactly  by  employing  higher-order  Lie  derivatives  and  the 
ascending  chain  property  of  Noetherian  rings.  A  Lie  deriva¬ 
tive  of  a  polynomial  p  :  Rn  — >  R  with  respect  to  the  polyno¬ 
mial  vector  held  f  :  R™  — ►  Rn  is  also  a  polynomial  denoted 
£f (p)  and  defined  as  £f(p)  =  Vp  ■  f  =  X)”=1  §jhfi-  It  gives 
the  total  derivative  of  the  p  with  respect  to  time,  i.e.  the 
rate  of  change  of  p  along  the  solutions  to  the  corresponding 
system  of  ODEs.  Higher-order  Lie  derivatives  are  defined 
inductively  as  £j  (p)  =  £f(£^_1(p)),  with  (p)  =  p. 

In  addition  to  [25],  a  description  of  the  main  idea  behind 


the  procedure  for  constructing  Inf(S)  may  be  found  in  17 


Section  5.4] ;  a  brief  sketch  of  this  construction  is  also  given  in 
Appendix  [A]  of  this  article.  Similar  ideas  employing  higher- 
order  Lie  derivatives  and  ascending  chains  of  ideals  have 
also  appeared  elsewhere,  e.g.  [331  [16] .  As  a  consequence, 


4 A  number  of  algorithms  have  been  developed  since 
Tarski’s  [38]  and  Seidenberg’s  [37]  seminal  papers,  e.g.  the 
CAD  algorithm  due  to  Collins  6F 
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the  sets  Entei'f(S),  Exitf(S)  and  Bouncef(S)  are  also  semi- 
algebraic  and  may  be  computed  exactly  using  a  terminating 
algorithm. 


5.  DYNAMIC  PROPERTIES  OF  OPERA¬ 
TIONAL  MODELS 

This  section  will  illustrate  some  of  the  dynamic  phenom¬ 
ena  observed  in  the  operational  models  that  we  can  compute 
using  Algorithm  [l]  and  will  discuss  some  of  the  differences 
in  their  behaviour  when  different  types  of  guard  conditions 
are  employed. 


5.1  Non-determinism 

Non-determinism  occurs  when  the  piecewise-smooth  sys¬ 
tem  may  evolve  inside  more  than  one  of  its  modes.  At  first 
sight,  this  may  look  surprising  because  in  a  PWS  any  state 
x  £  M  belongs  to  exactly  one  region  Mi,  i  £  {1, . . . ,  m}, 
if  one  indeed  has  a  mathematical  partition  of  M  into  these 
regions,  and  therefore  there  cannot  be  any  ambiguity  in  the 
choice  of  the  ODEs  that  should  govern  the  continuous  state 
evolution  at  x.  However,  generalized  solutions  to  the  system 
at  x  may  not  be  unique  even  when  the  ODEs  inside  each 
mode  all  have  unique  solutions  when  considered  separately. 
This  is  mirrored  in  our  operational  models,  where  we  aug¬ 
ment  the  regions  Mi  with  their  respective  “entry”  and  “exit” 
sets  to  obtain  the  augmented  mode  invariants  ©T  in  the  hy¬ 
brid  automaton.  One  may  face  a  scenario  where  x  £  ©t; 
and  x  £  ©Tj,  with  i  ^  j,  and  both  transition  guards  be¬ 
tween  the  two  states  qi  and  qj  are  enabled.  For  instance, 
x  may  lie  in  a  region  where  both  ©l;  A  ©b,  A  In^  (Mi)  and 
©h  A  ©b,  A  In^  ( Mj )  hold  true. 

The  standard  semantics  of  transition  guards  of  hybrid  au¬ 
tomata  is  that  they  enable  transitions,  but  do  not  force  them 
(this  is  known  as  non-urgent,  or  may  semantics  13).  Thus, 
while  there  is  no  ambiguity  about  the  initial  discrete  state 
of  the  hybrid  automaton  for  any  given  x  £  M,  the  system  is 
free  to  take  an  enabled  transition  immediately  after  it  starts 
evolving.  This  non-determinism  can  be  informally  under¬ 
stood  as  capturing  the  “instability”  that  arbitrarily  small 
perturbations  in  the  initial  state  can  cause  in  the  mode 
switching  behaviour  of  the  piecewise-smooth  system. 


5.2  Chattering  Runs 

A  phenomenon  known  as  chatter  is  traditionally  associ¬ 
ated  with  so-called  Zeno  behaviour  that  can  occur  in  mathe¬ 
matical  models  of  hybrid  systems  and  can  present  a  problem 
for  their  simulation  and  verification.  This  behaviour  is  non¬ 
physical  and  manifests  itself  in  the  possibility  of  performing 
an  infinite  number  of  transition  in  a  finite  amount  of  time. 

For  example,  a  hybrid  automaton  will  admit  chattering 
runs  whenever  for  two  distinct  states  and  qj  there  are 
transitions  in  both  directions  such  that  their  respective  tran¬ 
sition  guards  have  non-empty  intersection.  Any  state  x 
within  this  intersection  can  shuttle  back  and  forth  between 
the  states  qi  and  qj  an  arbitrary  (though  perhaps  not  infi¬ 
nite)  number  of  times. 


As  an  example,  let  us  consider  a  PWS  with  two  modes: 


x  =  fi(a;) 


±i=0, 

±2  =  *2  +  2  , 


X  =  f2(x) 


*1  =  *1  +  4*2  —  *1*2  , 

±2  =  *2  —  Xl  +  2  , 


*1  <  0  , 


*i  >  0. 


By  running  Algorithm  [T]  with  guard  conditions  of  type  I, 
one  obtains  a  hybrid  automaton  shown  in  Fig.  |2b|  This  au¬ 
tomaton  admits  chattering  runs  because  on  the  set  charac¬ 
terized  by  *i  =  0A*2  >  0  the  guards  for  transitions  between 
both  modes  are  enabled  simultaneously  and  the  system  may 
thus  shuttle  back  and  forth  arbitrarily  may  times  without 
advancing  in  (continuous)  time.  However,  if  one  were  to 
employ  guard  conditions  of  type  II,  the  resulting  automaton 
(Fig-  [3}  would  be  chatter-free. 


IP 

m 

2  4 

Xl 

(a)  Phase  portrait. 


(b)  Chattering  automaton. 


Figure  2:  Chattering  in  the  presence  of  non-determinism. 


ii  =  0Ai2>0 


/  qi  \ 

J  Q2  \ 

(  x  =  fi(a;)  ) 

X  =  f2(x)  J 

V  xi  <  0  / 

V  xi  >  0  J 

x\  =  0  A  X2  <  0 

Figure  3: 

Chatter-free  automaton. 

Since  infinite  Zeno  executions  cannot  in  practice  be  real¬ 
ized,  it  is  common  to  consider  only  the  non-Zeno  executions 
when  modelling  systems  using  hybrid  automata  [22||lf]  (this 
is  also  the  case  with  hybrid  programs  [35| ) . 

We  should  note  that  infinite  chattering  runs  are  a  spe¬ 
cial  kind  of  Zeno  behaviour,  which  some  authors  distinguish 
from  the  more  involved  genuine  Zeno  behaviour  (see  e.g.  [2]). 
Chatter-free  automata  may  still  suffer  from  this  latter  type 
of  Zeno  behaviour.  Detecting  and  eliminating  genuine  Zeno 
behaviour  in  hybrid  automata  is  highly  non-trivial  and  is 
the  focus  of  ongoing  research. 


5.3  Mythical  Modes 

A  piecewise-smooth  system  may  feature  a  mode  Mi  inside 
which  it  is  altogether  impossible  to  evolve  continuously  ac¬ 
cording  to  its  respective  system  of  ODEs  x  =  f;(a;).  More 
precisely,  it  is  possible  that  Mi  H  In^  (Mi)  =  0.  Inside  such 
a  mode,  the  (continuous)  state  of  the  system  remains  invari¬ 
ant  and  may  only  change  by  switching  into  a  different  mode; 
such  a  mode  is  sometimes  called  mythical  30]  31  .  For  ex¬ 
ample,  in  a  system  where  the  state  space  is  the  real  line  R 
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that  is  partitioned  into  3  modes  x  <  0,  x  =  0  and  x  >  0 
where  the  dynamics  is  respectively  x  =  1,  x  =  2  and  x  =  3, 
the  mode  x  =  0  is  mythical. 


Figure  4:  Mythical  mode  q2. 


Following  our  approach,  the  mode  invariants  for  the  hy¬ 
brid  automaton  are  augmented  to  be  x  <  0,  x  =  0  and 
x  >  0  respectively,  and  a  transition  from  x  <  0  into  x  >  0 
is  possible  without  ever  visiting  the  mythical  mode.  In  gen¬ 
eral,  in  hybrid  automata  constructed  using  our  method  (e.g. 
Fig  .[4]where  only  possible  transitions  are  depicted  with  their 
guards)  it  is  impossible  to  transition  into  mythical  modes 
with  any  of  the  three  types  of  guard  conditions. 

5.4  Sliding  Modes 

In  control  systems  literature,  it  is  not  uncommon  to  en¬ 
counter  systems  of  the  form 

™  =  /  fiO)  s(x)  >°. 

\  f2{x)  s(x)  <  0, 

where  s  :  R™  — >  R  is  some  differentiable  (often  polyno¬ 
mial)  function.  These  and  similar  systems  are  sometimes 
termed  variable  structure  systems  (VSS)  and  have  been  ap¬ 
plied  in  discontinuous  non-linear  control  strategies,  known 
as  variable  structure  control  (VSC).  A  phenomenon  known 
as  sliding  motion  lies  at  the  heart  of  an  important  class 
of  variable  structure  control,  known  as  sliding  mode  con¬ 
trol  (SMC),  which,  broadly  speaking,  achieves  system  order 
reduction  by  steering  the  trajectories  of  an  n-dimensional 
system  onto  an  n  —  1  dimensional  switching  hyper-surface 
in  the  system’s  state  space,  defined  by  s  =  0.  The  so-called 
sliding  motion  taking  place  on  the  hyper-surface  corresponds 
to  the  infinitely-fast  switching  between  the  modes  governing 
the  evolution  on  either  side  of  the  surface  [45],  i.e.  inside 
regions  where  s  >  0  and  s  <  0. 


Remark  3.  Note  however,  that  the  description  of  the  sys¬ 
tem  may  not  explicitly  prescribe  any  dynamics  on  the  switch¬ 
ing  surface  s  =  0  itself. 


In  practice,  sliding  motions  are  often  modelled  by  introduc¬ 
ing  a  so-called  equivalent  control  41  on  the  switching  sur¬ 
face;  this  is  achieved  by  letting 


X  =  f,(x)  =  Mtt)+f2(tt)+Me9fl(*)-f2(3Q 


,  (s)  +  £f2  (s)  ,  ,,  , 

where  ueq  =  -V — 1—. - — - — -r— ,  be  the  sliding  dynamics  on 

£f2(s)  - 

the  surface  s  =  0  (e.g.  see  32] ). 

Let  us  consider  a  2-dimensional  non-linear  system  with  a 
1-dimensional  sliding  mode  that  was  obtained  by  applying 


Figure  5:  Piecewise-smooth  system  x  =  5(ai)  with  a  sliding 
mode  at  x2  =  0  that  is  unstable  when  xi  >  0  (shown  in  red) 
and  a  stable  when  xi  <  0  (in  green). 

Sliding  occurs  on  the  set  characterized  by  x2  =  0  and 
x  =  (2(2:)  is  the  equivalent  control  dynamics  which  steers 
the  system  along  the  surface  *2  =  0  (Fig.  |5a|.  The  system 
exhibits  both  stable  and  unstable  sliding  behaviour,  which 
can  be  observed  in  the  phase  portrait,  as  shown  in  Fig.  |5b| 
Roughly  speaking,  in  the  neighbourhoods  of  states  where  the 
sliding  mode  is  stable  the  vector  fields  are  “pointing  towards” 
the  sliding  set,  whereas  in  the  neighbourhood  of  states  where 
it  is  unstable  the  vector  fields  are  “pointing  outwards”  away 
from  the  set. 

For  this  system,  different  types  of  guard  conditions  lead  to 
radically  different  operational  models.  The  resulting  hybrid 
automata  employing  guard  conditions  of  type  I,  II  and  III 
are  respectively  shown  in  Fig.  E  Fig- 0  and  Fig.  [8] 


Figure  6:  Hybrid  automaton  model  with  guard  conditions 
of  type  I. 
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Figure  7:  Hybrid  automaton  model  with  guard  conditions 
of  type  II. 


Figure  8:  Hybrid  automaton  model  with  guard  conditions 
of  type  III. 


The  three  automata  differ  in  the  way  they  model  non¬ 
determinism  in  the  system.  In  particular  guard  conditions 
of  type  III  result  in  the  automaton  in  Fig.  [8]  which  is  com¬ 
pletely  deterministic  and  only  models  the  stable  sliding  tak¬ 
ing  place  in  the  system;  there  is  no  non-determinism  corre¬ 
sponding  to  unstable  sliding  in  this  operational  model.  In 
practice,  this  behaviour  is  un-physical  because  unstable  mo¬ 
tions  can  leave  the  unstable  sliding  mode  under  arbitrarily 
small  perturbations  in  the  state  or  the  vector  field.  As  such, 
this  operational  model  represents  a  mathematical  idealiza¬ 
tion  which  is  of  little  use  when  modelling  physical  systems. 
However,  if  physical  considerations  are  unimportant,  the 
model  is  interesting  because  it  has  the  property  that  discrete 
transitions  are  taken  precisely  when  they  are  enabled,  in  a 
way  that  is  analogous  to  some  non-standard  urgent /must 
semantics  for  transition  guards  of  hybrid  automata. 

The  hybrid  automaton  in  Fig.  [7]  models  both  stable  and 
unstable  sliding  and  is  additionally  chatter-free,  whereas  the 
automaton  in  Fig.[6]admits  chattering  runs  when  the  contin¬ 
uous  state  is  at  the  origin.  Of  all  these  operational  models, 
the  one  employing  guard  conditions  of  type  II  (in  Fig.  [7|  is 
perhaps  the  most  physically  meaningful  and  faithful  to  the 
intended  behaviour  of  the  system. 

6.  OUTLOOK  AND  RELATED  WORK 

Having  automatic  means  of  computing  operational  mod¬ 
els  of  systems  which  can  be  concisely  specified  (but  whose 
operational  models  require  an  unreasonable  amount  of  effort 
and  care  to  explicitly  write  down  manually)  is  a  significant 
enabling  factor.  In  general,  computing  adequate  hybrid  au¬ 
tomaton  models  of  systems  is  highly  non-trivial  [29/  The 
examples  used  in  this  paper  are  very  simple  and  are  intended 
to  highlight  differences  between  the  different  models;  more 
interesting  examples  of  PWS  lead  to  automata  that  are  in¬ 
deed  quite  formidable.  We  have  implemented  our  HA  syn¬ 
thesis  algorithm  in  Mathematica  and  are  able  to  generate  au¬ 


tomata  in  the  format  of  the  verification  tool  SpaceEx  |jl4] 

The  hybrid  automata  we  are  able  to  generate  can  provide 
suitable  models  for  addressing  the  problem  of  verification 
(e.g.  of  safety  and  liveness  properties)  and  benefit  from  a 
large  and  growing  number  of  software  tools  developed  to 
verify  or  simulate  hybrid  systems  14]  |24[  [5|  |15[  |43] .  Verifi¬ 
cation  technology  for  hybrid  systems  has  improved  tremen¬ 
dously  in  the  last  two  decades;  however,  in  much  of  existing 
work  there  are  significant  restrictions  on  the  form  of  hybrid 
automata,  such  as  e.g.  only  allowing  linear  ODEs  to  govern 
continuous  evolution,  or  only  allowing  a  specific  class  of  sets 
(e.g.  polytopes)  to  act  as  mode  invariants  for  the  states  of 
the  automaton.  We  should  note  that  in  this  sense  the  class 
of  systems  considered  in  this  paper  is  very  broad  because  it 
allows  for  non-linear  continuous  dynamics  and  for  arbitrary 
semi-algebraic  sets  to  act  as  mode  invariants  and  transition 
guards. 

It  is  our  hope  our  techniques  will  in  future  be  applied  to 
modelling  and  verification  of  properties  in  systems  with  engi¬ 
neering  applications  that  employ  variable  structure  control. 
We  stress,  however,  that  many  important  questions  remain 
unresolved.  For  instance,  the  difficult  task  of  categorizing 
and  classifying  the  possible  kinds  of  operational  models  (be¬ 
yond  the  three  presented)  remains  to  be  addressed.  Interest¬ 
ing  questions  as  to  which  of  the  many  possible  types  of  oper¬ 
ational  semantics  for  PWS  that  can  be  obtained  through  us¬ 
ing  techniques  described  in  this  paper  are  “physically  mean¬ 
ingful”  (and  for  what  phenomena)  present  many  intriguing 
avenues  for  future  research. 


6.1  Related  Work 

Lygeros  et  al.  studied  existence  and  uniqueness  of  exe¬ 
cutions  of  hybrid  automata  in  [26] ,  giving  conditions  under 
which  hybrid  automata  are  deterministic  and  non-blocking. 
We  note  that  there  are  important  differences  in  definitions, 
e.g.  the  use  of  semi-open  time  intervals  in  [26],  such  as  in 

Out(gi)  =  {x  €  R"  |  V  e.  3  t  G  [0,e).  ifit(x)  V  Mi}, 

where  Mi  =  Dom(qi).  This  differs  from  definitions  used  in 
this  paper,  e.g. 


-ilnf^Mj)  =  {*  G  M"  |  V  e.  3  t  G  (0,  e).  <pt(x)  V  Mi}. 

Remark  4.  Similar  notions  also  exist  in  the  ODE  litera¬ 
ture,  e.g.  “ingress”  and  “egress”  sets  used  to  state  and  prove 
the  Wazewski  principle  (\21\  p.  282],  &)■ 

The  work  in  [26  is  also  similar  in  using  Lie  derivatives  of 
functions  to  reason  about  the  transition  behaviour;  however, 
the  authors  consider  a  special  class  of  hybrid  automata  in 
which  mode  invariants  can  be  characterized  by  sub-level  sets 
of  analytic  functions,  i.e.  cr(x)  >  0.  The  same  restriction 
was  used  in  the  work  of  Johansson  et  al.  [23]  and  already 
rules  out  systems  in  which  mode  invariants  are  given  by 
polytopes.  We  work  under  much  more  general  assumptions 
where  the  mode  invariants  are  semi-algebraic  sets  and  work 
with  their  representations  directly.  Further  investigations  of 
existence  and  uniqueness  of  executions  of  hybrid  automata 
were  reported  in  [27|. 

5 An  implementation  is  available  from  http://www.lix. 
polytechnique.fr/~ghorbal/EMSOFT17 
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7.  CONCLUSION 

In  this  paper  we  presented  a  methodology  for  automat¬ 
ically  synthesizing  hybrid  automata  from  descriptions  of 
piecewise-smooth  polynomial  systems,  i.e.  systems  of  dis¬ 
continuous  ODEs  that  are  polynomial  on  disjoint  semi- 
algebraic  sets  forming  a  partition  of  the  state  space.  The 
hybrid  automata  thus  obtained  provide  operational  models 
of  piecewise-smooth  systems,  which  can  behave  in  different 
ways,  depending  on  certain  choices  in  formulating  the  con¬ 
ditions  on  the  transition  guards.  We  have  described  in  Sec¬ 
tions  [OJ  [473]  three  alternative  choices  that  can  be  exercised 
in  this  regard,  and  which  can  be  thought  of  as  giving  differ¬ 
ent  operational  meaning  (i.e.  semantics)  to  the  piecewise- 
smooth  systems.  Many  more  choices  are  possible  and  the 
task  of  studying  and  classifying  these  possibilities  presents 
a  very  interesting  direction  for  further  research. 

One  of  our  main  aims  in  this  paper  was  to  present  a  case  as 
to  why  it  is  not  meaningful  to  speak  of  “the  hybrid  automa¬ 
ton  model”  of  a  given  piecewise-smooth  system  without  a 
precise  description  of  how  the  said  hybrid  automaton  model 
was  created.  We  argue  that  a  synthesis  algorithm,  such  as 
that  presented  in  Section  [4. 2[  is  needed  in  order  to  provide 
this  description. 

We  believe  that  correct  modelling  of  piecewise-smooth  sys¬ 
tems  is  a  problem  that  is  of  more  than  just  theoretical  in¬ 
terest,  since  systems  of  this  type  occur  frequently  in  control 
engineering  (often  in  the  context  of  autonomous  switching 
or  sliding  mode  controllers).  Their  representation  as  differ¬ 
ential  equations  active  inside  certain  designated  regions  is 
deceptively  simple  and  great  care  needs  to  be  taken  when 
extracting  operational  models  from  these  simple  representa¬ 
tions.  Our  work  addressed  some  of  the  fundamental  difficul¬ 
ties  inherent  in  this  task. 
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APPENDIX 

A.  COMPUTING  “IN  SETS”  EXACTLY 

To  give  an  idea  of  how  Iiif(S)  is  computed  exactly,  con¬ 
sider  a  set  S  which  is  given  by  p  <  0,  where  p  is  some  poly¬ 
nomial  function  in  the  state  variables  xi, . ..  ,x„  with  real 
coefficients.  Firstly,  note  that  each  point  x  in  the  interior 
of  S,  i.e.  satisfying  p  <  0  necessarily  lies  inside  Iiif  (p  <  0) 
because  motion  within  the  interior  is  always  possible  within 
some  open  neighbourhood.  The  set  p  <  0  thus  provides  the 
first  under-approximation  of  the  set  Inf(p  <  0).  We  now  re¬ 
fine  this  under-approximation  by  adding  some  of  the  states 
satisfying  p  =  0,  for  which  a  sufficient  (but  not  necessary) 
condition  for  membership  in  Inj(p  <  0)  is  that  of  satisfy¬ 
ing  the  inequality  £f(p)  <  0.  This  is  intuitive  because  the 
rate  of  change  of  p  at  such  a  state  is  negative  and  therefore 
the  system  will  immediately  evolve  into  the  set  satisfying 
p  <  0.  However,  for  states  satisfying  p  =  0  and  £f(p)  =  0, 
one  needs  to  check  that  the  second-order  Lie  derivative  is 
negative  (i.e.  £j?  (p)  <  0)  in  order  to  conclude  their  member¬ 
ship  in  Irif(p  <  0),  and  so  on  for  higher-order  Lie  derivatives. 
Intuitively,  these  cases  correspond  to  situations  where  “the 
velocity  is  zero,  but  the  acceleration  is  negative”,  etc.,  which 
likewise  ensures  that  the  system  cannot  evolve  into  a  state 
satisfying  p  >  0  (i.e.  the  complement  of  p  <  0)  immedi¬ 
ately  afterwards.  The  set  Irif(p  <  0)  is  then  constructed  as 
follows: 

Illf  (p  <  0)  =  p  <  0 

V  (p  =  0  A  £f(p)  <  0) 

V  (p  =  0  A  £j(p)  =  0  A  £j  (p)  <  0) 


V  (p  =  0  A  £f(p)  =  0  A  •  •  •  A  £f*(p)  <  0) 

The  fact  that  the  number  k  is  finite  and  can  be  computed 
is  a  consequence  of  Hilbert’s  basis  theorem  and  the  ascend¬ 
ing  chain  property  of  Noetherian  rings  (see  e.g.  28  Sec. 
2.3.2]).  These  fundamental  results  guarantee  that  one  is 
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always  able  to  find  a  k  £  N  such  that  the  ideal  member¬ 
ship  £[v  ( p )  £  (p,  £f(p), . . . ,  £f(p))  holds  for  all  K  >  k. 
This  property  is  equivalent  to  the  statement  that  for  each 
K  >  k  the  following  equality  holds:  £fr(p)  =  ctoP  + 
ai£f(p)  +  -  ■  -+ak£f{p),  where  the  coefficients  ao,  on, . . .  ,ak 
are  some  polynomials  in  the  ring  R[aq, . . . ,  £„].  Thus,  when¬ 
ever  p  =  £f(p)  =  ■  •  ■  =  £j  (p)  =  0  holds,  one  necessarily  has 
(p)  =  0  for  all  K  >  0,  and  thus  it  is  impossible  to  grow 
the  under-approximation  of  Inj(p  <  0)  by  adding  any  more 
disjuncts  of  the  form  p  =  0  A  £f(p)  =  0  A  ■  •  ■  A  £j  (p)  = 
0  A  •  ■  ■  A  £^  (p)  <  0  for  any  K  >  k  and  the  construction  is 
therefore  complete.  In  practice,  the  number  k  is  computed 
using  Grobner  bases  (e.g.  see  [9j  Chap.  2]). 


B.  ENTER,  EXIT  AND  BOUNCE  SETS 

Consider  a  semi-algebraic  set  described  by  the  formula 
S  =  x i  +  (x2  +  3)2  <  6  A  —3  <  X2  and  let  the  dynamics  of 
the  system,  x  =  f(a;),  be  given  by  the  system  of  polynomial 
ODEs:  xi  =  aqa;!  —  1,3:2  =  —  aq.  Fig.  9a  shows  the  set  S 


*1 


(a)  Semi-algebraic  set  ScR2 


(b)  Enter j(S)  and  Exitj(S) 


Figure  9:  Semi-algebraic  set,  along  with  its  “entry”  states 
(in  green)  and  “exit”  states  (in  red). 


along  with  some  of  the  trajectories  of  the  system.  The  set 
of  “entering  states”,  given  by 

Enterf(5)  =  (aq  +  3  =  0Aii  <  0  A  a:?  +  a;|  +  63:2  +  3  <  0) 

V  (x2  +  3>0Ai|  +  i2  +  63:2  +  3  =  0  A  aqa;2  <  *1  (£2  +  4)), 
is  shown  in  green  in  Fig.  |9b|  and 

Exitf(S)  =  ^0  <  £1  <  V6  A  £2  +  3  =  0^  V  (£2  +  3  >  0 

A  £2  +  £2  (£2  +  6)  +  3  =  0  A  £2£2  >  £1  (£2  +  4) ) 

is  shown  in  red.  Note  that  these  two  sets  need  not  necessarily 
include  all  the  points  on  the  boundary  of  S.  The  black 
points  in  Fig.  [9b] represent  states  on  the  boundary  which  are 
neither  in  Enter^(5)  nor  Exitf(S).  In  particular,  Bounce^S) 
includes  the  point  at  the  centre  of  the  semi-circle,  i.e.  £1  = 
0A£2  =  —3,  whereas  the  remaining  three  points  in  the  figure 
belong  to  Bouncef(-'S). 


Gi.e.  £f(p)  is  in  the  ideal  generated  by  the  finite  set  of 
polynomials  {p,  £f  (p), . . . ,  £ffe  (p)} 
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Abstract  We  propose  a  method  for  verifying  persistence  of  nonlinear  hybrid  sys¬ 
tems.  Given  some  system  and  an  initial  set  of  states,  the  method  can  guarantee 
that  system  trajectories  always  eventually  evolve  into  some  specified  target  sub¬ 
set  of  the  states  of  one  of  the  discrete  modes  of  the  system,  and  always  remain 
within  this  target  region.  The  method  also  computes  a  time-bound  within  which 
the  target  region  is  always  reached.  The  approach  combines  flow-pipe  compu¬ 
tation  with  deductive  reasoning  about  invariants  and  is  more  general  than  each 
technique  alone.  We  illustrate  the  method  with  a  case  study  concerning  showing 
that  potentially  destructive  stick-slip  oscillations  of  an  oil-well  drill  eventually 
die  away  for  a  certain  choice  of  drill  control  parameters.  The  case  study  demon¬ 
strates  how  just  using  flow-pipes  or  just  reasoning  about  invariants  alone  can  be 
insufficient.  The  case  study  also  nicely  shows  the  richness  of  systems  that  the 
method  can  handle:  the  case  study  features  a  mode  with  non-polynomial  (nonlin¬ 
ear)  ODEs  and  we  manage  to  prove  the  persistence  property  with  the  aid  of  an 
automatic  prover  specifically  designed  for  handling  transcendental  functions. 

1  Introduction 

Hybrid  systems  combine  discrete  and  continuous  behaviour  and  provide  a  very  gen¬ 
eral  framework  for  modelling  and  analyzing  the  behaviour  of  systems  such  as  those 
implemented  in  modern  embedded  control  software.  Although  a  number  of  tools  and 
methods  have  been  developed  for  verifying  properties  of  hybrid  systems,  most  are 
geared  towards  proving  bounded-time  safety  properties,  often  employing  set  reachabil¬ 
ity  computations  based  on  constructing  over-approximating  enclosures  of  the  reachable 
states  of  ordinary  differential  equations  (e.g.  171141131211).  Methods  capable  of  proving 
unbounded-time  safety  properties  often  rely  (explicitly  or  otherwise)  on  constructing 
continuous  invariants  (e.g.  I421B1.  and  referred  to  in  short  as  invariants).  Such  invari¬ 
ants  may  be  thought  of  as  a  generalization  of  positively  invariant  sets  (see  e.g.  0)  and 
which  are  analogous  to  inductive  invariants  used  in  computer  science  to  reason  about 
the  correctness  of  discrete  programs  using  Hoare  logic. 

*  This  material  is  based  upon  work  supported  by  the  UK  Engineering  and  Physical  Sciences 
Research  Council  under  grants  EPSRC  EP/I010335/1  and  EP/J001058/1,  the  National  Science 
Foundation  (NSF)  under  grant  numbers  CNS  1464311  and  CCF  1527398,  the  Air  Force  Re¬ 
search  Laboratory  (AFRL)  through  contract  number  FA8750-15-1-0105,  and  the  Air  Force 
Office  of  Scientific  Research  (AFOSR)  under  contract  number  FA9550-15-1-0258. 
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We  argue  in  this  paper  that  a  combined  approach  employing  bounded  time  reach¬ 
ability  analysis  and  reasoning  about  invariants  can  be  effective  in  proving  persistence 
and  safety  properties  in  non-polynomial  (nonlinear)  hybrid  systems.  We  illustrate  the 
combined  approach  using  a  detailed  case  study  with  non-polynomial  ODEs  for  which 
neither  approach  individually  was  sufficient  to  establish  the  desired  safety  and  persis¬ 
tence  properties. 

Methods  for  bounded  time  safety  verification  cannot  in  general  be  applied  to  prove 
safety  for  all  time  and  their  accuracy  tends  to  degrade  for  large  time  bounds,  especially 
for  nonlinear  systems.  Verification  using  invariants,  while  a  powerful  technique  that  can 
prove  strong  properties  about  nonlinear  systems,  relies  on  the  ability  to  find  invariants 
that  are  sufficient  for  proving  the  unbounded  time  safety  property.  In  practice,  many 
invariants  for  the  system  can  be  found  which  fall  short  of  this  requirement,  often  for  the 
simple  reason  that  they  do  not  include  all  the  initial  states  of  the  system.  We  show  how 
a  combined  approach  employing  both  verification  methods  can,  in  some  cases,  address 
these  limitations. 

Contributions. 

In  this  paper  we  (I)  show  that  bounded  time  safety  verification  based  on  flowpipe  con¬ 
struction  can  be  naturally  combined  with  invariants  to  verify  persistence  and  unbounded 
time  safety  properties,  addressing  some  of  the  limitations  of  each  verification  method 
when  considered  in  isolation.  (II)  To  illustrate  the  approach,  we  consider  a  simplified 
torsional  model  of  a  conventional  oil  well  drill  string  that  has  been  the  subject  of  nu¬ 
merous  studies  by  Navarro-Lopez  et  al.  l34l.  (Ill)  We  discuss  some  of  the  challenges 
that  currently  stand  in  the  way  of  fully  automatic  verification  using  this  approach.  Ad¬ 
ditionally,  we  provide  a  readable  overview  of  the  methods  employed  in  the  verification 
process  and  the  obstacles  that  present  themselves  when  these  methods  are  applied  in 
practice. 

2  Safety  and  Persistence  for  Hybrid  Automata 

2.1  Preliminaries 

A  number  of  formalisms  exist  for  specifying  hybrid  systems.  The  most  popular  frame¬ 
work  at  present  is  that  of  hybrid  automata  |3|19|,  which  are  essentially  discrete  tran¬ 
sition  systems  in  which  each  discrete  state  represents  an  operating  mode  inside  which 
the  system  evolves  continuously  according  to  an  ODE  under  some  evolution  constraint. 
Additionally,  transition  guards  and  reset  maps  are  used  to  specify  the  discrete  transition 
behaviour  (i.e.  switching)  between  the  operating  modes.  A  sketch  of  the  syntax  and 
semantics  of  hybrid  automata  is  as  follows. 

Definition  1  (Hybrid  automaton  [26]).  Formally,  a  hybrid  automaton  is  given  by 
(■ Q ,  Var,  f,  Init,  Inv,  T,  G,  R),  where 

•  Q  =  (go,  qi,  ■  ■  ■ ,  qk]  is  a  finite  set  of  discrete  states  (modes), 

•  Var  =  {xi,X2, . . . ,  xn}  is  a  finite  set  of  continuous  variables, 

•  f  :  Q  x  Rn  — ¥  Rn  gives  the  vector  field  defining  continuous  evolution  inside  each 
mode, 

•  Init  C  Q  x  r  is  the  set  of  initial  states. 
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•  Inv  :  Q  — >  2®  gives  the  mode  invariants  constraining  evolution  for  every  discrete 
state, 

•  T  C  Q  x  Q  is  the  transition  relation, 

•  G  :  T  — y  2®  gives  the  guard  conditions  for  enabling  transitions, 

•  R  :T  — >■  2®"  xR"  gives  the  reset  map. 

A  hybrid  state  of  the  automaton  is  of  the  form  (q,  x)  £  Q  x  Rn.  A  hybrid  time  trajectory 
is  a  sequence  (which  may  be  finite  or  infinite)  of  intervals  r  =  { hflLo,  for  which 
Ii  =  [t,;  t.']  for  all  /  <  N  and  n  <  r,'  =  73+1  for  all  i.  If  the  sequence  is  finite,  then  either 
In  =  [tjv,t^]  or  In  =  [tn,t'n).  Intuitively,  one  may  think  of  r,  as  the  times  at  which 
discrete  transitions  occur.  An  execution  (or  a  run  or  trajectory )  of  a  hybrid  automaton 
defined  to  be  (t,  q,  ipl(x)),  where  r  is  a  hybrid  time  trajectory,  q  :  (r)  — x  Q  (where  (r) 
is  defined  to  be  the  set  {0, 1, ,  N}  if  r  is  finite  and  {0, 1, ...  }  otherwise)  and  <p\(x) 
is  a  collection  of  diffeomorphisms  tpl(x)  :  Ii  — x  Rn  such  that  (r/(0),  v?o(a;))  £  Init,  for 
all  t  £  x  =  f(q(i),ipl(x))  and  pl(x)  £  Inv(i).  For  all  i  £  (r)  \  {Ar}  it  is  also 

required  that  transitions  respect  the  guards  and  reset  maps,  i.e.  e  =  (q(i),q(i  +  1))  £  T, 
Plfx)  £  G(e)  and  «;(*)»  ^n+^a:))  £  R{e). 

We  consider  MTL^formulas  satisfied  by  trajectories.  The  satisfaction  relation  is  of 
form  p  |=p  f,  read  as  “ trajectory  p  at  position  p  satisfies  temporal  logic  formula  j>\ 
where  positions  on  a  trajectory  are  identified  by  pairs  of  form  (i,  t)  where  i  <  N  and 
time  t  £  It-  We  use  the  MTL  modality  □/<)>  which  states  that  formula  <i>  always  holds 
in  time  interval  I  in  the  future.  Formally,  this  can  be  defined  as  p  |=p  □/<?!>  =  Mp'  > 
p  s.t.  (p'.2  —  p.  2)  £  I.  p  \=p'  f,  where  (*',  t ')  >  (i,t)  =i'  >  iV  (*'  =  i  A  f  >t).  Similarly 
we  can  define  the  modality  O if  which  states  that  formula  <!>  eventually  holds  at  some 
time  in  the  time  interval  I  in  the  future.  An  MTL  formula  is  valid  for  a  given  hybrid 
automaton  if  it  is  satisfied  by  all  trajectories  of  that  automaton  starting  at  position  (0, 0). 
For  clarity  when  writing  MTL  formulas,  we  assume  trajectories  are  not  restricted  to 
start  in  Init  states  and  instead  introduce  Init  predicates  into  the  formulas  when  we  want 
restrictions. 

Alternative  formalisms  for  hybrid  systems,  such  as  hybrid  programs  ED,  enjoy  the 
property  of  having  a  compositional  semantics  and  can  be  used  to  verify  properties  of 
systems  by  verifying  properties  of  their  parts  in  a  theorem  prover  11441151 .  Other  formal 
modelling  frameworks  for  hybrid  systems,  such  as  Hybrid  CSP  f24l.  have  also  found 
application  in  theorem  provers  I60l62l . 

2.2  Bounded  Time  Safety  and  Eventuality 

The  bounded-time  safety  verification  problem  (with  some  finite  time  bound  t  >  0)  is 
concerned  with  establishing  that  given  an  initial  set  of  states  Init  CQxI”  and  a  set 
of  safe  states  Safe  CQx  Rn,  the  state  of  the  system  may  not  leave  Safe  within  time  t 
along  any  valid  trajectory  r  of  the  system.  In  the  absence  of  closed-form  solutions  to 
the  ODEs,  this  property  may  be  established  by  verified  integration,  i.e.  by  computing 
successive  over-approximating  enclosures  (known  as  flowpipes)  of  the  reachable  states 
in  discrete  time  steps.  Bounded-time  reachability  analysis  can  be  extended  to  full  hy¬ 
brid  systems  by  also  computing/over-approximating  the  discrete  reachable  states  (up  to 
some  finite  bound  on  the  number  of  discrete  transitions). 

3  Metric  Temporal  Logic;  see  e.g.  (22). 
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A  number  of  bounded-time  verification  tools  for  hybrid  systems  have  been  devel¬ 
oped  based  on  verified  integration  using  interval  enclosures.  For  instance,  iSAT-ODE ,  a 
verification  tool  for  hybrid  systems  developed  by  Eggers  et  al.  ca  relies  on  the  verified 
integration  tool  VNODE-LP  by  Nedialkov  071  for  computing  the  enclosures.  Other  ex¬ 
amples  include  dReach ,  a  reachability  analysis  tool  for  hybrid  systems  developed  by 
Kong  et  al.  GU,  which  uses  the  CAPD  library  QJ.  Over-approximating  enclosures  can 
in  practice  be  very  precise  for  small  time  horizons,  but  tend  to  become  conservative 
when  the  time  bound  is  large  (due  to  the  so-called  wrapping  effect,  which  is  a  problem 
caused  by  the  successive  build-up  of  over-approximation  errors  that  arises  in  interval- 
based  methods;  see  e.g.  11381.)  An  alternative  verified  integration  method  using  Taylor 
models  was  introduced  by  Makino  and  Berz  (see  038!)  and  can  address  some  of  these 
drawbacks,  often  providing  tighter  enclosures  of  the  reachable  set.  Implementations 
of  the  method  have  been  reported  in  COSY  INFINITY ,  a  scientific  computing  tool  by 
Makino  and  Berz  l29l ;  VSPODE,  a  tool  for  computing  validated  solutions  to  parametric 
ODEs  by  Lin  and  Stadtherr  |23l;  and  in  Flow*,  a  bounded-time  verification  for  hybrid 
systems  developed  by  Chen  et  al.  0. 

Because  flowpipes  provide  an  over-approximation  of  the  reachable  states  at  a  given 
time,  verified  integration  using  flowpipes  can  also  be  used  to  reason  about  liveness 
properties  such  as  eventuality,  i.e.  when  a  system  is  guaranteed  to  eventually  enter  some 
target  set  having  started  off  at  some  point  in  an  initial  set.  The  bounded-time  safety  and 
eventuality  properties  may  be  more  concisely  expressed  by  using  MTL  notation,  i.e.  by 
writing  Init  — >  n[o,t]  Safe,  and  Init  — t  O[o,t]  Target,  where  Init  describes  the  initial  set  of 
states.  Safe  C  Q  x  Rn  is  the  set  of  safe  states  and  Target  C  Q  x  I"  is  the  target  region 
which  is  to  be  eventually  attained. 

Remark  2.  The  bounded  time  eventuality  properties  we  consider  in  this  paper  are  more 
restrictive  than  the  general  (unbounded  time)  case.  For  instance,  consider  a  continuous 
2-dimensional  system  governed  by  ii  =  X2,  £2  =  0  and  confined  to  evolve  in  the  region 
where  *2  >  0.  If  one  starts  this  system  inside  a  state  where  xi  =  0,  it  will  eventually 
evolve  into  a  state  where  xi  =  1  by  following  the  solution,  however  one  may  not  put  a 
finite  bound  on  the  time  for  this  to  happen.  Thus,  while  x\  =  0  — >■  O[o,<x>)  =  1  is  true 

for  this  system  the  bounded  time  eventuality  property  xi  =  0  — ►  O[o,t]  *1  =  1,  will  not 
hold  for  any  finite  t  >  0. 

2.3  Unbounded  Time  Safety 

A  safety  property  for  unbounded  time  may  be  more  concisely  expressed  using  an  MTL 
formula: 

Init  □[o.oo)  Safe. 

A  proof  of  such  a  safety  assertion  is  most  commonly  achieved  by  finding  an  appropri¬ 
ate  invariant,  I  C  Q  x  Rn,  which  contains  no  unsafe  states  (i.e.  I  C  Safe)  and  such 
that  the  state  of  the  system  may  not  escape  from  I  into  an  unsafe  state  along  any  valid 
trajectory  of  the  system.  Invariance  is  a  special  kind  of  safety  assertion  and  may  be 
written  as  I  — >■  □[o,00)  I.  A  number  of  techniques  have  been  developed  for  proving  in¬ 
variance  properties  for  continuous  systems  without  the  need  to  compute  solutions  to  the 
ODEs  149141 158I25I17I531. 
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2.4  Combining  Unbounded  Time  Safety  with  Eventuality  to  Prove  Persistence 

In  linear  temporal  logic,  a  persistence  property  states  that  a  formula  is  ‘eventually  al¬ 
ways’  true.  For  instance,  using  persistence  one  may  express  the  property  that  a  system 
starting  in  any  initial  state  always  eventually  reaches  some  target  set  and  then  always 
stays  within  this  set.  Using  MTL  notation,  we  can  write  this  as: 

Init  -J-  Op.cx,)  □(o.oo)  Target. 

Persistence  properties  generalize  the  concept  of  stability.  With  stability  one  is  concerned 
with  showing  that  the  state  of  a  system  always  converges  to  some  particular  equilibrium 
point.  With  persistence,  one  only  requires  that  the  system  state  eventually  becomes 
always  trapped  within  some  set  of  states. 

In  this  paper  we  are  concerned  with  a  slightly  stronger  form  of  persistence,  where 
one  ensures  that  the  target  set  is  always  reached  within  some  specified  time  t: 


Init  -»  O[o, t]  0(0,00)  Target. 


We  observe  that  a  way  of  proving  this  is  to  find  a  set  I  C  Target  such  that: 

1.  Init  — ►  O[0, t]  I  holds,  and 

2.  I  is  an  invariant  for  the  system. 

This  fact  can  be  stated  more  formally  as  a  rule  of  inference: 


(Persistence) 


Init  ->■  O[o, t]  I  I  -»  0(0,00)  I  I  ->  Target 
Init  ->•  O(o, t]  O[o,oo)  Target 


Previous  Sections  2.2  and  2.3  respectively  surveyed  how  the  eventuality  premise  Init  — ► 
Op, t]  I  and  invariant  premise  I  — >  O[o,oo)  I  can  be  established  by  a  variety  of  automated 
techniques.  In  Section  5  we  explore  automation  challenges  further  and  remark  on  on¬ 
going  work  addressing  how  to  automatically  generate  suitable  invariants  I. 

2.5  Using  Persistence  to  Prove  Safety 


Finding  appropriate  invariants  to  prove  unbounded  time  safety  as  explained  above  in 
Section [23]can  in  practice  be  very  difficult.  It  might  be  the  case  that  invariants  I  C  Safe 
for  the  system  can  be  found,  but  also  ensuring  that  Init  C  I  is  infeasible.  Nevertheless  it 
might  be  the  case  that  one  of  these  invariants  I  is  always  eventually  reached  by  trajec¬ 
tories  starting  in  Init  and  all  those  trajectories  are  contained  within  Safe.  In  such  cases. 
Safe  is  indeed  a  safety  property  of  the  system  when  starting  from  any  point  in  Init.  More 
precisely,  if  one  can  find  an  invariant  I  as  explained  above  in  Section  2.4  to  show  the 
persistence  property:  Init  — >  O[0,t]  n[o,cx>)  Safe,  and  further  one  can  show  for  the  same 
time  bound  t  that:  Init  — ►  n[o,t]  Safe,  then  one  has:  Init  — >  □(o.oo)  Safe.  As  a  result,  one 
may  potentially  utilize  invariants  that  were  by  themselves  insufficient  for  proving  the 
safety  property. 


Remark  3.  The  problem  of  showing  that  a  state  satisfying  □(o,oo)  Safe  is  reached  in  finite 
time  t,  while  ensuring  that  the  formula  1=1  [o.fi  Safe  also  holds  (i.e.  states  satisfying  -iSafe 
are  avoided  up  to  time  t)  is  sometimes  called  a  reach-avoid  problem  ED. 
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Even  if  one’s  goal  is  to  establish  bounded-time  rather  than  unbounded-time  safety 
properties,  this  inference  scheme  could  still  be  of  use,  as  it  could  significantly  reduce 
the  time  bound  t  needed  for  bounded  time  reachability  analysis.  In  practice,  successive 
over-approximation  of  the  reachable  states  using  flowpipes  tends  to  become  conserva¬ 
tive  for  large  values  of  t.  In  highly  non-linear  systems  one  can  realistically  expect  to 
compute  flowpipes  only  for  very  modest  time  bounds  (e.g.  in  chaotic  systems  flowpipes 
are  guaranteed  to  ‘blow  up’,  but  invariants  may  still  sometimes  be  found).  Instead,  it 
may  in  some  cases  be  possible  to  prove  the  safety  property  by  computing  flowpipes  up 
to  some  small  time  bound,  after  which  the  system  can  be  shown  to  be  inside  an  invariant 
that  implies  the  safety  property  for  all  times  thereafter. 

3  An  example  persistence  verification  problem 

Stick-slip  oscillations  are  commonly  encountered  in  mechanical  engineering  in  the  con¬ 
text  of  modelling  the  effects  of  dynamic  friction.  Informally,  the  phenomenon  manifests 
itself  in  the  system  becoming  “stuck”  and  “unstuck”  repeatedly,  which  results  in  un¬ 
steady  “jerky”  motions.  In  engineering  practice,  stick-slip  oscillations  can  often  degrade 
performance  and  cause  failures  when  operating  expensive  machinery  ll36l.  Although  the 
problem  of  demonstrating  absence  of  stick-slip  oscillations  in  a  system  is  primarily  mo¬ 
tivated  by  safety  considerations,  it  would  be  misleading  to  call  this  a  safety  verification 
problem.  Instead,  the  problem  may  broadly  be  described  as  that  of  demonstrating  that 
the  system  (in  finite  time)  enters  a  state  in  which  no  stick-slip  motion  is  possible  and 
remains  there  indefinitely.  Using  MTL  one  may  write: 


Init  ->•  O[0,t]  □[o,oo)  Steady, 


where  Steady  describes  the  states  in  which  harmful  oscillations  cannot  occur.  The  for¬ 
mula  may  informally  be  read  as  saying  that  “from  any  initial  configuration,  the  system 
will  eventually  evolve  within  time  t  into  a  state  region  where  it  is  always  steady”. 

As  an  example  of  a  system  in  which  eventual  absence  of  stick-slip  oscillations  is 
important,  we  consider  a  well-studied  f34l  model  of  a  simplified  conventional  oil  well 
drill  string.  The  system  can  be  characterized  in  terms  of  the  following  variables:  ipr, 
the  angular  displacement  of  the  top  rotary  system;  <pb,  the  angular  displacement  of  the 
drilling  bit;  <fir,  the  angular  velocity  of  the  top  rotary  system;  and  fib,  the  angular  velocity 
of  the  drilling  bit.  The  continuous  state  of  the  system  x(t)  e  R‘!  can  be  described  in 
terms  of  these  variables,  i.e.  x(t)  =  (fir,  —  <fib,  fib)T ■  The  system  has  two  control 
parameters:  W„b  giving  the  weight  applied  on  the  drilling  bit,  and  u  =  Tm  giving  the 
surface  motor  torque.  The  dynamics  is  governed  a  non-linear  system  of  ODEs  x  —  f(x), 
given  by: 


r 


(1) 


X2  =  Xl—  X3, 


(2) 


(3) 
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The  term  Tfb( X3)  denotes  the  friction  modelling  the  bit-rock  contact  and  is  responsible 
for  the  non-polynomial  non-linearity.  It  is  given  by 

/  _Z£.|a;3|\ 

WobRb\^iJ-cb  +  0^  -  Mcje  "S  Jsgn(a;3), 

where  sgnja^)  =  A-J  if  i3  ^  0  and  sgn^)  e  [—1, 1]  if  *3  =  0.  Constants  used  in  the 
model  l34ll  are  as  follows:  <7 ,  =  50Nms/rad,  kt  =  861.5336  Nm/rad,  Jr  =  2212  kg  m2, 
Jb  =  471.9698  kgm2,  Rb  =  0.155575  m,  ct  =  172.3067  Nms/rad,  cr  =  425Nms/rad, 
Hcb  =  0.5,  Hsb  =  0.8,  76  =  0.9,  uf  =  1  rad/s.  Even  though  at  first  glance  the  system 
looks  like  a  plain  continuous  system  with  a  single  set  of  differential  equations,  it  is 
effectively  a  hybrid  system  with  at  least  3  modes,  where  the  drilling  bit  is:  “rotating 
forward”  (:r3  >  0),  “stopped”  (x3  =  0),  and  “rotating  backward”  (*3  <  0).  A  sub-mode 
of  the  stopped  mode  models  when  the  drill  bit  is  stuck.  In  this  sub-mode,  the  torque 
components  on  the  drill  bit  due  to  ct,  Cb  and  kt  are  insufficient  to  overcome  the  static 
friction  W0bRbPcb,  and  sgn(x3)  is  further  constrained  so  as  to  ensure  0:3  =  0. 

Once  the  drill  is  in  operation,  so-called  stick-slip  oscillations  can  cause  damage 
when  the  bit  repeatedly  becomes  stuck  and  unstuck  due  to  friction  in  the  bottom  hole 
assembly.  In  the  model  this  behaviour  would  correspond  to  the  system  entering  a  state 
where  13  =  0  repeatedly.  The  objective  is  to  verify  the  eventual  absence  of  stick-slip 
oscillations  in  the  system  initialised  at  the  origin  (i.e.  at  rest)  for  some  given  choice  of 
the  control  parameters  W0b  and  u.  Previous  work  by  Navarro-Lopez  and  Carter  (34]  ex¬ 
plored  modelling  the  simplified  model  of  the  drill  as  a  hybrid  automaton  and  simulated 
the  resulting  models  in  Stateflow  and  Modelica. 

Bit  angular  velocity  (rad/s)  Bit  angular  velocity  (rad/s) 


(a)  Stick-slip  motion  (undesirable)  (b)  Stabilization  (desired  behaviour) 

Figure  1 :  Simulations  can  exhibit  stabilization  with  positive  bit  angular  velocity  and 
stick-slip  bit  motion. 

Simulations,  such  as  those  obtained  in  (34],  using  different  models  and  control  pa¬ 
rameters  for  the  drill  can  suggest  stick-slip  oscillations  or  their  absence  (illustrated  in 
Fig.[T]i  in  a  particular  model,  however  the  task  of  verifying  their  eventual  absence  can¬ 
not  be  adequately  addressed  with  simulation  alone.  In  practice  however,  simulation  is 
incredibly  useful  in  providing  some  degree  of  confidence  in  the  overall  result,  which  is 
very  important  to  know  before  attempting  verification. 

A  simulation  of  the  system  with  a  concrete  choice  for  the  control  parameters  Wab  = 
50, 000  N  and  u  =  6, 000  Nm,  shown  as  a  trajectory  in  the  3-dimensional  state  space 
in  Fig  [3a]  suggests  that  the  system  does  not  exhibit  stick-slip  oscillations,  because  the 
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trajectory  is  observed  to  start  at  the  origin,  escape  the  surface  (*3  =  ()£]  and  stabilize 
around  a  point  where  the  angular  velocity  of  the  drilling  bit  is  positive  (*3  >  0). 

4  Verifying  Persistence 

The  property  of  interest,  i.e.  the  eventual  absence  of  stick-slip  oscillation  that  we  ob¬ 
serve  in  the  simulation,  may  be  phrased  as  the  following  formula  in  metric  temporal 
logic:  n=0Aa:2  =  0Ai3  =  0^  O[0,t]  n[o,oo)  *3  >  0,  which  informally  asserts  that  the 
system  initialised  at  the  origin  will  eventually  (diamond  modality)  enter  a  state  where  it 
is  always  (box  modality)  the  case  that  *3  >  0.  In  the  following  sections  we  describe  a 
method  for  proving  this  assertion.  Following  our  approach,  we  break  the  problem  down 
into  the  following  two  sub-problems: 

1.  Finding  an  appropriate  invariant  I  in  which  the  property  *3  >  0  holds.  For 
this  we  employ  continuous/positive  invariants,  discussed  in  the  next  section. 

2.  Proving  that  the  system  reaches  a  state  in  the  set  I  in  finite  time  when  initialised  at 
the  origin,  i.e.  n=OAi2  =  0Ai3  =  0->  O[0;i]  /.[?] 

4.1  Continuous  Invariant 

Finding  continuous  invariants  that  are  sufficient  to  guarantee  a  given  property  is  in  prac¬ 
tice  remarkably  difficult.  Methods  for  automatic  continuous  invariant  generation  have 
been  reported  by  numerous  authors  1I49I59I18I53I52I25I631 161301541.  but  in  practice  of¬ 
ten  result  in  “coarse”  invariants  that  cannot  be  used  to  prove  the  property  of  interest,  or 
require  an  unreasonable  amount  of  time  due  to  their  reliance  on  expensive  real  quantifier 
elimination  algorithms. 

Stability  analysis  (involving  a  linearisation;  see  [|56j  for  details)  can  be  used  to  sug¬ 
gest  a  polynomial  function  V  :  Rn  — >■  R,  given  by 

V(x)  =  50599.6  -  14235.7*1  +  1234.22*?  -  4351.43*2  +  342.329*i*2 

+  288.032*|  -  3865.81*3  +  367.657*i*3  +  18.2594*2*3  +  241.37*5, 

for  which  we  can  reasonably  conjecture  that  V(x)  <  1400  defines  a  positively  invariant 
set  under  the  flow  of  our  non-linear  system.  Geometrically,  this  represents  an  ellipsoid 
that  lies  above  the  surface  defined  by  *3  =  0  in  the  state  space  (see  Fig.[3bj>.  In  order  to 
prove  the  invariance  property,  it  is  sufficient  to  show  that  the  following  holds  0 

V*  GR3.  V{x)  =  1400  ->  VC-  f{x)<  0.  (4) 

Unfortunately,  in  the  presence  of  non-polynomial  terms]?]  a  first  order  sentence  will  in 
general  not  belong  to  a  decidable  theory  j5TJ,  although  there  has  recently  been  progress 
in  broadening  the  scope  of  the  popular  CAD  algorithm  a  for  real  quantifier  elimination 
to  work  with  restricted  classes  of  non-polynomial  problems  l57l. 

4  The  system  exhibits  sliding  behaviour  on  a  portion  of  this  surface  known  as  the  sliding  set. 

See  1541. 

5  Files  for  the  case  study  are  available  online,  http  :  / /www .  veri vital .  com/nfm2017 

6  Here  V  denotes  the  gradient  of  V,  i.e.  the  vector  of  partial  derivatives  (JjA-," . . . , 

7  E.g.  those  featured  in  the  right-hand  side  of  the  ODE,  i.e.  f(x). 
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In  practice,  this  conjecture  is  easily  proved  in  under  5  seconds  using  MetiTarski,  an 
automatic  theorem  prover,  developed  by  L.C.  Paulson  and  co-workers  at  the  University 
of  Cambridge,  designed  specifically  for  proving  universally  quantified  first  order  con¬ 
jectures  featuring  transcendental  functions  (such  as  sin, cos,  In,  exp,  etc.)  The  interested 
reader  may  find  more  details  about  the  MetiTarski  system  in  121401. 

Remark  4.  Although  Wolfram’s  Mathematica  10  computer  algebra  system  also  pro¬ 
vides  some  functionality  for  proving  first-order  conjectures  featuring  non-polynomial 
expressions  using  its  Reduce  [  ]  function,  we  were  unable  (on  our  systeirQi  to  prove 
conjecture  <[4]»  this  way  after  over  an  hour  of  computation,  after  which  the  Mathematica 
kernel  crashed. 

The  automatic  proof  of  conjecture  (j4ji  obtained  using  MetiTarski  (provided  we  trust 
the  system)  establishes  that  V ( x )  <  1400  defines  a  positively  invariant  set,  and  thus  we 
are  guaranteed  that  solutions  initialised  inside  this  set  remain  there  at  all  future  times. 
In  order  to  be  certain  that  no  outgoing  discrete  transitions  of  the  hybrid  system  are 
possible  when  the  system  is  evolving  inside  V (x)  <  1400,  we  further  require  a  proof  of 
the  following  conjecture  featuring  only  polynomial  terms: 

V  x  £  R3.  V(x)  <  1400  — >  X3  >  0.  (5) 


An  automatic  proof  of  this  conjecture  may  be  obtained  using  an  implementation  of  a 
decision  procedure  for  first-order  real  arithmetic. 

4.2  Verified  Integration 


In  order  to  show  that  the  system  does  indeed  enter  the  positively  invariant  ellipsoid 
V(x)  <  1400  in  finite  time,  it  is  not  sufficient  to  observe  this  in  a  simulation  (as  in 
Fig.  3b  i,  which  is  why  we  use  a  tool  employing  verified  integration  based  on  Taylor 
models.  Flow*  (implemented  by  Chen  et  al.  Q)  is  a  bounded-time  safety  verification 
tool  for  hybrid  systems  that  computes  Taylor  models  to  analyze  continuous  reachability. 
The  tool  works  by  computing  successive  over-approximations  (flowpipes)  of  the  reach¬ 
able  set  of  the  system,  which  are  internally  represented  using  Taylor  models  (but  which 
may  in  turn  be  over-approximated  by  a  bounding  hyper-box  and  easily  rendered). 

Fig.  2a  shows  the  bounding  boxes  of  solution  enclosures  computed  from  the  point 
initial  condition  at  the  origin  using  Flow*  with  adaptive  time  steps  and  Taylor  models  of 
order  13,  a  time  bound  of  12.7  and  the  same  control  parameters  used  in  the  simulation 
(i.e.  u  =  6, 000  Nm,  Wab  =  50, 000  N).  We  observe  that  once  solutions  escape  to  the 
region  where  x3  >  0,  they  maintain  a  positive  x:{  component  for  the  duration  of  the  time 
bound. 

The  last  flowpipe  computed  by  Flow*  for  this  problem  can  be  bounded  inside  the 
hyper-rectangle  BoundBox  characterized  by  the  formula 

t-,  39  „  „  51  ^  26  7  ,  37 

BoundBox  =  —  <ii<4A  -  <  x2  <  —  A  -  <  x3  <  —. 


Once  more,  using  a  decision  procedure  for  real  arithmetic,  we  can  check  that  the  fol¬ 
lowing  sentence  is  true: 


V  x  £  R3.  BoundBox  —r  V  (x)  <  1400. 

Intel  i5-2520M  CPU  @  2.50GHz,  4GB  RAM,  running  Arch  Linux  kernel  4.2.5- 1. 
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(a)  Verified  integration  up  to  time  t  =  12.7  (b)  Verified  integration  up  to  time  t  =  12.2 

from  a  point  initial  condition  at  the  origin.  from  an  interval  initial  condition. 

Figure  2:  Verified  integration  using  Flow*. 

If  we  are  able  to  establish  the  following  facts: 

1.  I  — y  □[o,oo)  I  (I  is  a  continuous  invariant), 

2.  I  — y  Steady  (inside  /,  there  are  no  harmful  oscillations),  and 

3.  Init  —y  O[o,t]  I  (the  system  enters  the  region  /  in  finite  time), 

then  we  can  conclude  that  Init  —y  O[o,t]  O[o,oo)  Steady  is  also  true  and  the  system  does 
not  exhibit  harmful  stick-slip  oscillations  when  started  inside  Init.  By  taking  Init  to 
be  the  origin  x\  =  0  A  12  =  0  A  *3  =  0,  I  to  be  the  positively  invariant  sub-level  set 
V ( x )  <  1400  and  Steady  to  be  *3  >  0,  we  are  able  to  conclude  the  temporal  property: 

ii=0Ai2=0Ai3=0->  O[0,t]  □[t.oo)  *3  >  0. 
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Verified  integration  using  Taylor  models  also  allows  us  to  consider  sets  of  possible 
initial  conditions,  rather  than  initial  points  (illustrated  in  Fig.  2b  i.  This  is  useful  when 
there  is  uncertainty  about  the  system’s  initial  configuration;  however,  in  practice  this 
comes  with  a  significant  performance  overhead  for  verified  integration. 


(a)  Simulation  showing  stabilization  with  posi-  (b)  Simulation  showing  eventual  entry  into  an 
tive  bit  angular  velocity.  ellipsoidal  invariant. 

Figure  3:  Simulation  of  the  hybrid  system  initialised  at  the  origin  with  Wab  =  50, 000  N 
and  u  =  6000  Nm.  The  trajectory  is  contained  by  the  flowpipes  shown  in  Fig. [2a] and  is 
observed  to  enter  the  positively  invariant  ellipsoid  V(x)  <  1400,  illustrating  the  persis¬ 
tence  property  of  eventual  absence  of  stick-slip  oscillations. 

5  Outlook  and  Challenges  to  Automation 


Correctness  of  reachability  analysis  tools  based  on  verified  integration  is  a  soundness 
critical  to  the  overall  verification  approach,  which  makes  for  a  strong  case  in  favour  of 
using  formally  verified  implementations.  At  present  few  are  available,  e.g.  see  recent 
work  by  Immler  EOll  which  presented  a  formally  verified  continuous  reachability  al¬ 
gorithm  based  on  adaptive  Runge-Kutta  methods.  Verified  implementations  of  Taylor 
model-based  reachability  analysis  algorithms  for  continuous  and  hybrid  systems  would 
clearly  be  very  valuable.  One  alternative  to  over-approximating  reachable  sets  of  con¬ 
tinuous  systems  using  flowpipes  is  based  on  simulating  the  system  using  a  finite  set  of 
sampling  trajectories  and  employs  sensitivity  analysis  to  address  the  coverage  problem. 
This  technique  was  explored  by  Donze  and  Maler  in  flOl.  A  similar  approach  employ¬ 
ing  matrix  measures  has  more  recently  been  studied  by  Maidens  and  Arcak  1281271. 

As  an  alternative  to  using  verified  integration,  a  number  of  deductive  methods 
are  available  for  proving  eventuality  properties  in  continuous  and  hybrid  systems 
(e.g.  1421551).  These  approaches  can  be  much  more  powerful  since  they  allow  one  to 
work  with  more  general  classes  of  initial  and  target  regions  that  are  necessarily  out  of 
scope  for  methods  based  on  verified  integration  (e.g.  they  can  work  with  initial  sets  that 
are  unbounded,  disconnected,  etc.)  Making  effective  use  of  the  deductive  verification 
tools  currently  in  existence  typically  requires  significant  input  and  expertise  on  part  of 
the  user  (finding  the  right  invariants  being  one  of  the  major  stumbling  blocks  in  prac¬ 
tice),  in  stark  contrast  to  the  near-complete  level  of  automation  offered  by  tools  based 
on  verified  integration.  Methods  for  automatic  continuous  invariant  generation  are  cru- 
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cial  to  the  mechanization  of  the  overall  verification  approach.  Progress  on  this  problem 
would  be  hugely  enabling  for  non-experts  and  specialists  alike,  as  it  would  relieve  them 
from  the  task  of  manually  constructing  appropriate  invariants,  which  often  requires  in¬ 
tuition  and  expertise.  Work  in  this  area  is  ongoing  (see  e.g.  1431251541).  Indeed,  progress 
on  this  problem  is  also  crucial  to  providing  a  greater  level  of  automation  in  deductive 
verification  tools. 

6  Related  Work 

Combining  elements  of  qualitative  and  quantitative  reasoning  to  study  the  behaviour 
of  dynamical  systems  has  previously  been  explored  in  the  case  of  planar  systems  by 
Nishida  et  al.  (39).  The  idea  of  combining  bounded-time  reachability  analysis  with 
qualitative  analysis  in  the  form  of  discrete  abstraction  was  investigated  by  Clarke  et  al. 
in  ©.  Similar  ideas  are  employed  by  Carter  (6)  and  Navarro -Lopez  in  1351,  where  the 
concept  of  deadness  is  introduced  and  used  as  a  way  of  disproving  liveness  properties. 
Intuitively,  deadness  is  a  formalization  of  an  idea  that  inside  certain  regions  the  system 
cannot  be  live,  i.e.  some  desired  property  may  never  become  true  as  the  system  evolves 
inside  a  “deadness  region”.  These  ideas  were  used  in  a  case  study  0  Chapter  5]  also 
featuring  the  drill  system  studied  in  (34),  but  with  a  different  set  of  control  parameters 
and  in  which  the  verification  objective  was  to  prove  the  existence  of  a  single  trajectory 
for  which  the  drill  eventually  gets  “stuck”,  which  is  sufficient  to  disprove  the  liveness 
(oscillation)  property. 

Region  stability  is  similar  to  our  notion  of  persistence  (45).  which  requires  all  tra¬ 
jectories  to  eventually  reach  some  region  of  the  state  space.  Sound  and  complete  proof 
rules  for  establishing  region  stability  have  been  explored  and  automated  (47),  as  have 
more  efficient  encodings  of  the  proof  rule  that  scale  better  in  dimensionality  ED-  How¬ 
ever,  all  algorithms  we  are  aware  of  for  checking  region  stability  require  linear  or  sim¬ 
pler  (timed  or  rectangular)  ODEs  H45I47B46I13 1111 1481 .  Strong  attractors  are  basins  of 
attraction  where  every  state  in  the  state  space  eventually  reaches  a  region  of  the  state 
space  (45).  Some  algorithms  do  not  check  region  stability,  but  actually  check  stronger 
properties  such  as  strong  attraction,  that  imply  region  stability  (45).  In  contrast  to  these 
works,  our  method  checks  the  weaker  notion  of  persistence  for  nonlinear  ODEs. 

She  and  Ratschan  studied  methods  of  proving  set  eventuality  in  continuous  sys¬ 
tems  under  constraints  using  Lyapunov-like  functions  (50).  Duggirala  and  Mitra  also 
employed  Lyapunov-like  function  concepts  to  prove  inevitability  properties  in  hybrid 
systems  (33-  Mohlmann  et  al.  developed  Stabhyil  (33),  which  can  be  applied  to  non¬ 
linear  hybrid  systems  and  checks  classical  notions  of  Lyapunov  stability,  which  is  a 
strictly  stronger  property  than  persistence.  In  (32)  Mohlmann  et  al.  extended  their  work 
and  applied  similar  ideas,  using  information  about  (necessarily  invariant)  sub-level  sets 
of  Lyapunov  functions  to  terminate  reachability  analysis  used  for  safety  verification. 
Prabhakar  and  Soto  have  explored  abstractions  that  enable  proving  stability  properties 
without  having  to  search  for  Lyapunov  functions,  albeit  these  are  not  currently  applica¬ 
ble  to  nonlinear  systems  (48).  In  summary,  in  contrast  to  other  works  listed  above,  our 
approach  enables  proving  persistence  properties  in  conjunction  with  safety  properties 

9  e.g  numerical  solution  computation  with  “qualitative”  features,  such  as  invariance  of  certain 
regions. 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 

281 


Verifying  safety  and  persistence  properties  of  hybrid  systems 


13 


for  nonlinear,  non-polynomial  hybrid  systems  and  does  not  put  restrictions  on  the  form 
or  the  type  of  the  invariant  used  in  conjunction  with  bounded  time  reachability  analysis. 

7  Conclusion 

This  paper  explored  a  combined  technique  for  safety  and  persistence  verification  em¬ 
ploying  continuous  invariants  and  reachable  set  computation  based  on  constructing 
flowpipes.  The  approach  was  illustrated  on  a  model  of  a  simplified  oil  well  drill  string 
system  studied  by  Navarro-Lopez  et  al.,  where  the  verification  objective  is  to  prove  ab¬ 
sence  of  damaging  stick-slip  oscillations.  The  system  was  useful  in  highlighting  many 
of  the  existing  practical  challenges  to  applying  and  automating  the  proposed  verifica¬ 
tion  method.  Many  competing  approaches  already  exist  for  verifying  safety  in  hybrid 
systems,  but  these  rarely  combine  different  methods  for  reachability  analysis  and  de¬ 
ductive  verification,  which  our  approach  combines.  We  demonstrate  that  a  combination 
of  different  approaches  can  be  more  practically  useful  than  each  constituent  approach 
taken  in  isolation. 
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Abstract 

This  benchmark  suite  is  composed  of  nine  examples  of  large-scale  lin¬ 
ear  systems,  ranging  in  dimensionality  in  the  tens  to  the  low  thousands. 
The  benchmarks  are  derived  from  diverse  fields  such  as  civil  engineering 
and  robotics,  and  are  based  on  similar  existing  test  sets  for  model-order 
reduction  algorithms  in  control  and  numerical  analysis.  Each  example  is 
provided  in  the  SpaceEx  XML  model  format  as  single-mode  hybrid  au¬ 
tomaton  and  are  compatible  with  the  HyST  model  transformation  tool  to 
support  analysis  in  other  verification  tools.  Some  preliminary  reachability 
analysis  results  for  some  of  the  smaller  examples  (on  the  order  of  tens  of 
dimensions)  are  presented  using  SpaceEx. 

Category:  academic  Difficulty:  low  through  challenge 


1  Context  and  Origins 

Symbolic  state-space  analysis  has  shown  advantages  in  safety  verification  of  con¬ 
tinuous  and  hybrid  systems  in  which  the  essential  task  is  computing  the  set  of 
reachable  states  symbolically  with  an  iterative  algorithm  [1],  The  main  challenge 
of  this  approach  is  state-space  explosion,  which  roughly  is  that  the  complexity 
of  computation  grows  exponentially  with  the  system  dimensionality  [2] .  To  im¬ 
plement  efficiently  symbolic  reachability  algorithms,  significant  effort  has  been 
invested  in  finding  appropriate  representations  for  the  set  of  states  that  supports 
efficient  operations  used  in  the  iterative  computation.  From  classical  polyhe¬ 
dral  representations  which  are  used  in  hybrid  systems  model  checkers  such  as 
HyTech  [3,4]  and  d/dt  [5],  more  efficient  representations  such  as  zonotopes  [6-8] 
and  support  functions  [9,10]  have  been  proposed  and  integrated  in  tools  such  as 
CORA  and  SpaceEx  that  use  these  state-of-the-art  representations  for  analysis 
of  hybrid  systems  with  linear  dynamics. 
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No. 

Benchmark 

Type 

n 

m 

P 

1 

Motor  control  system  (MCS) 

LTI 

8 

2 

2 

2 

Building  model  (BM)  [12] 

LTI 

48 

1 

1 

3 

International  space  station  (ISS)  [12] 

LTI 

270 

3 

3 

4 

Partial  differential  equation  (Pde)  [12] 

LTI 

84 

1 

1 

5 

FOM  [12] 

LTI 

1006 

1 

1 

6 

Modified  nodal  analysis  model  1  [12](MNA-1) 

LTI 

578 

9 

9 

7 

Modified  nodal  analysis  model  5  [12](MNA-5) 

LTI 

10913 

9 

9 

8 

Heat  equation  [12] 

LTI 

200 

1 

1 

9 

Clamped  beam  model  [12] 

LTI 

348 

1 

1 

Table  2.1:  Benchmarks  for  the  order-reduction  abstraction  method  in  which 
n  is  dimension  of  the  system;  m  and  p  are  the  number  of  inputs  and  outputs 
respectively. 


In  spite  of  these  advances,  reachability  analysis  of  large-scale  systems  with 
hundreds  to  thousands  of  dimensions  is  still  infeasible  even  for  linear  time  invari¬ 
ant  (LTI)  systems,  i.e. ,  without  any  discrete  switching  behavior.  It  is  important 
to  develop  new  techniques  and  tools  that  can  be  used  to  verify  the  safety  of 
such  high-dimensional  systems,  which  usually  exist  in  a  broad  range  of  fields 
and  applications  such  as  control  systems,  biological  systems,  analog  circuits, 
and  multi-agent  systems. 

To  help  test  and  evaluate  reachability  analysis  methods  and  tools  to  en¬ 
able  verification  of  high-dimensional  systems,  we  construct  a  set  of  benchmarks 
that  are  essentially  LTI  systems  arising  from  model  order  reduction  [11,12]. 
These  benchmarks,  which  are  models  of  practical  systems  in  different  fields, 
have  dimensions  varying  from  ten  to  thousands.  Each  benchmark  is  given  in 
the  SpaceEx  format  as  a  single-mode  hybrid  automaton  and  can  be  easily  trans¬ 
formed  to  other  formats  such  as  dReach  [13]  or  Flow*  [14]  using  the  HyST  model 
transformation  tool  [15].  Reachability  analysis  of  some  of  the  small  and  medium- 
size  benchmarks  (i.e.,  <  50  dimensions  )  are  presented.  These  benchmarks  may 
be  effective  to  test  and  evaluate  the  scalability  of  verification  approaches  when 
dealing  with  large-size  benchmarks  (i.e.  >  50  dimensions). 

2  Brief  descriptions 

Since  most  of  benchmarks  are  high-dimensional,  their  dynamic  equations  cannot 
be  presented  in  detail  in  this  paper.  We  refer  readers  to  [11,12]  for  for  further 
details  and  derivations,  as  well  as  our  provided  supplementary  material.1  The 

1The  benchmarks  are  available  online,  http://verivital.com/hyst/benchmark-large- 
scale/ 
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Benchmark 

Initial  set  of  states 

Xq  —  {rro  €  Rn|  lb(i)  <  xq (i)  <  ub(i),  1  <  i  <  n} 

Input  constraint 

U  —  [wi,  •  •  •  ,  Um]T 

Safety  specification 

y  —  biv  ,yp\T 

Motor  con¬ 
trol  system 

lb(i)  =  ub(i)  =  0,  i  =  2, 3, 4, 6,  7, 8, 

(6(2)  =  0.002,  m6(2)  =  0.0025, 

(6(3)  =  0.001,  m6(3)  =  0.0015. 

Ml  e  [0.16,  0.3], 

M2  e  [0.2,  0.4], 

unsafe  region: 

0.35  <  yi  <  0.4, 

0.45  <y2<  0.6. 

Building 

model 

lb(i)  =  0.0002,  ub(i)  =  0.00025,  1  <  i  <  10, 

(6(25)  =  -0.0001,  m6(25)  =  0.0001, 
lb(i)  =  ub(i)  =  0,  11  <  i  <  48,  i  ^  25. 

mi  e  [0.8,  i]. 

unsafe  region: 

0.006  <  yi 

Partial  dif¬ 
ferential 
equation 

lb(i)  —  0,  u6(i)  —  0,  1  <  i  <  64 

(6(i)  =  0.001,  ub(i)  =  0.0015,  64  <  i  <  80, 

lb(i)  =  -0.002,  ub(i)  =  -0.0015,  81  <  i  <  84. 

mi  e  [0.5,  i]. 

safe  region: 

yi  <12 

International 
space  station 

lb(i)  =  -0.0001,  ub(i)  =  0.0001,  1  <  i  <  270. 

mi  e  [0,  o.i], 

m2  e  [0.8,  l], 
m3  e  [0.9,  l]. 

Safe  region: 

-0.0005  <  y3  <  0.0005 

FOM 

lb(i)  =  -0.0001,  ub(i)  =  0.0001,  1  <  i  <  400 
lb(i)  =  0.0002,  ub(i)  =  0.00025,  401  <  i  <  800, 
lb(i)  =  0,  ub(i)  =0,  801  <  i  <  1006. 

Mi  6  [-1,  1]. 

safe  region: 

yi  <  45 

MNA-1 

lb(i)  =  0.001,  ub(i)  =  0.0015,  1  <  i  <  2 
lb(i)  =  0,  ub(i)  =0,  3  <  i  <  578, 

Ui  -  0.1,  1  <  i  <  5, 

m  -  0.2,  6  <  i  <  9. 

unsafe  region: 

yi  >  0.5 

MNA-5 

lb(i)  =  0.0002,  ub(i)  =  0.00025,  1  <  i  <  10 
lb(i)  =  0,  ub(i )  =  0,  11  <  i  <  10913, 

m  -  0.1,  1  <  i  <  5, 

Ui  —  0.2,  6  <  i  <  9. 

safe  region: 

yi  <  0.2,  y\  <  0.15 

Heat  equa¬ 
tion 

lb(i)  =  0.6,  ub(i)  =  0.625,  1  <  i  <  2 
lb(i)  =  0,  ub(i)  =  0,  3  <  i  <  200, 

ui  e  [—0.5,  0.5]. 

safe  region: 

v\  <  o.i 

Clamped 
beam  model 

lb(i)  =  0,  ub(i)  =0,  1  <  i  <  300 

lb(i)  =  0.0015,  ub(i)  =  0.002,  301  <  i  <  348, 

Mi  e  [0.2,  0.8]. 

unsafe  region: 

yi  >  1000 

Table  2.2:  Initial  states,  input  constraints  and  safety  specification  for  the  out¬ 
puts  of  the  benchmarks. 


general  form  of  the  dynamics  is: 

x(t )  =  Ax(t)  +  Bu(t) 
y{t)  =  Cx{t), 

where  x(t)  £  Rn  is  the  system  state,  y(t)  £  Rp  is  the  system  output,  u(t)  is  the 
control  input,  A  £  Rraxn,  B  £  Rnxm,  and  C  £  Rpxn. 

In  this  section,  we  introduce  briefly  these  benchmarks.  Table  2.1  summarizes 
names,  number  of  dimensions,  and  numbers  of  inputs  and  outputs  of  the  bench¬ 
marks.  The  initial  set  of  states,  input  constraints,  and  safety  specifications  of 
the  benchmarks  are  given  in  Table  2.2. 

Motor  control  system.  The  motor  control  system  benchmark  includes  two 
motors  that  are  controlled  synchronously.  Each  motor  has  a  local  controller  that 
is  designed  using  pole  placement  method  [16]  to  control  the  motor  to  satisfy:  1) 
the  overshoot  of  the  motor  position  is  less  than  16%;  2)  setting  time  is  less  than 
0.04  seconds;  3)  No  steady-state  error,  even  in  the  presence  of  a  step  disturbance 
input. 
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Building  model.  The  building  model  is  a  model  of  the  Los  Angeles  University 
Hospital  with  8  floors,  each  of  which  has  3  degrees  of  freedom  [11].  This  system 
has  48  state  variables  in  which  we  are  mostly  interested  in  the  twenty-fifth  state 
X25  (f),  which  is  the  motion  of  the  first  coordinate.  The  twenty-fifth  state  is 
the  interested  output  of  the  building  model  and  should  not  reach  to  the  unsafe 
region  given  in  Table  2.2. 

Partial  differential  equation.  The  partial  differential  equation  (PDE)  is 
given  by 


dx  d2x  d2x  n„dx 

at  =  a?  +  S?  +  ~ 1801  +  /(”' 

where  x  is  a  function  of  time  t,  vertical  position  v  and  horizontal  position  z. 
This  problem  lies  on  a  square  domain  defined  by  two  opposite  points  (0,0) 
and  (1,1).  The  function  x(t,v,z)  is  zero  on  the  boundaries  of  the  square.  A 
state-space  equation  of  dimension  of  N  =  nvnz  of  this  PDE  can  be  given  by 
discretizing  with  centered  difference  approximation  on  a  grid  of  nv  x  nz  points. 
The  input  vector  corresponding  to  f(v,  z)  is  composed  of  random  elements  while 
the  output  vector  of  the  system  is  equated  to  the  input  vector  for  simplicity. 
The  state-space  model  of  PDE  covered  in  this  paper  corresponds  to  the  case  of 
nv  =  7  and  nz  =  12. 

International  Space  Station  (ISS).  The  ISS  state-space  model  presented 
in  this  paper  is  a  structural  model  of  component  1R  (Russian  service  module) 
of  the  International  Space  Station.  It  has  270  state  variables  with  three  inputs 
and  three  outputs. 

FOM.  This  is  state-space  model  of  a  dynamical  system  with  following  matri¬ 
ces: 


A  = 


A 1 


A2 


A 


i  Ai  — 


-1  100 

-100  -1 


>  A  2  — 


-1  200 

-200  -1 


A,  = 


-1  400 

-400  -1 


A4  = 


-1 


-2 


-1000 


Bt  =  C  =  [10---10  1— jj. 

6  1000 
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Modified  nodal  analysis  model.  The  following  Modified  Nodal  Analysis 
(MNA)  equation  is  constituted  from  connecting  voltage  sources  to  the  ports  of 
a  multiport: 


ExnA  =  Axn  +  Bup, 
ip  —  Cxn: 

in  which  ip  and  up  are  the  port  currents  and  voltages  vectors  respectively  and 


- N  -G 

,  E  = 

L 

0 

V 

A  = 

l 

C> 

4 

o 

1 _ 

0 

H 

i  — 

i 

where  v  and  i  are  variables  of  the  MNA  including  node  votages,  inductor  and 
voltage  source  currents,  respectively.  The  matrices  — A  and  E  represent  the 
conductance  and  susceptance  matrices.  The  matrices  —  N,  L  and  H  contains  the 
stamps  for  resistors,  capacitors  and  inductors,  respectively.  Matrix  G  consists 
of  1,  —1  and  0,  which  describe  the  current  variables  in  KirchhofF’s  Current  Law 
(KCL)  equation.  The  input  matrix  B  and  output  matrix  C  satisfy  B  =  CT . 
We  give  two  MNA  models  with  different  number  of  state  variables  in  the  paper. 

Heat  equation.  The  state-space  model  of  Heat  equation  is  giving  by  dis¬ 
cretizing  the  following  equation: 

f  PDE  T(x,t )  =  T(x,t)  +u(x,t),  x  e  (0,1);  t  >  0,j 

1  BCs  T{ 0,  t)  =  0  =  T(  1,  t),  t>  0,  f 

[ic  T(x,0)  =0,  xe  (0,1).  J 

where  T(x,t)  represents  the  temperature  field  on  a  thin  rod  and  u(x,t)  is  the 
heat  source. 

Clamped  beam  model.  The  state-space  clamped  beam  model,  which  is  ob¬ 
tained  by  spatial  discretization  of  an  appropriate  partial  different  equation,  has 
348  states,  one  input  and  one  output  in  which  the  input  represents  the  force 
applied  to  the  structure  and  the  output  is  the  displacement. 

3  Reachability  analysis 

Since  all  benchmarks  are  LT1  systems,  there  are  different  tools  that  can  be  used 
to  analyze  the  safety  of  these  benchmarks  such  as  SpaceEx  [10],  CORA  [17], 
CheckMake  [18],  DReach  [13],  and  Flow*  [14].  We  specify  each  benchmark  in 
the  SpaceEx  format  as  a  single-mode  hybrid  automaton,  which  can  be  easily 
transformed  to  other  formats  using  HyST  [15]. 

Table  3.1  presents  a  preliminary  overview  of  the  computation  cost  of  time- 
bounded  reachability  analysis  for  the  benchmarks  using  SpaceEx.  These  exper¬ 
iments  are  conducted  on  a  personal  computer  with  the  following  configurations: 
Intel  (R)  Core(TM)  i7-2677M  CPU  at  1.80GHz,  4GB  RAM,  and  64-bit  Win¬ 
dow  7.  The  reachability  analysis  is  conducted  in  a  bounded  time  range  [0,20s]. 
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Benchmark 

LGG 

STC 

Time(s) 

Time(s) 

Motor  control  system 

27 

N/A 

Building  model 

893 

N/A 

Partial  differential  equation 

OOT 

N/A 

International  space  station 

OOT 

N/A 

FOM 

OOT 

N/A 

MNA-1 

OOT 

N/A 

MNA-5 

OOT 

N/A 

Heat  equation 

OOT 

N/A 

Clamped  beam  model 

OOT 

N/A 

Table  3.1:  Computation  cost  for  verification  of  the  benchmarks  using 
SpaceEx  [10]  with  two  scenarios  LGG  [19]  and  STC  [20].  The  terms  of  “N/A” 
and  “OOT”  mean  “not  applicable”  and  “out  of  time”. 


The  SpaceEx  scenarios  tested  are  LGG  [19]  and  STC  [20].  The  sampling  time 
is  selected  as  0.001  for  all  benchmarks.  We  note  that  the  sampling  time  and 
time  horizon  should  be  selected  appropriately  based  on  the  dynamics  of  spe¬ 
cific  system,  for  example,  using  the  rule  of  thumb  to  pick  the  sampling  time 
based  on  the  inverse  of  the  maximum  eigenvalue.  Intuitively,  this  would  mean 
to  pick  large  sampling  times  for  slow  dynamics  and  small  sampling  times  for 
fast  dynamics.  Thus,  while  our  preliminary  results  as  shown  in  Table  3.1  indi¬ 
cate  some  examples  are  infeasible  for  analysis  with  SpaceEx,  it  is  possible  that 
a  more  careful  selection  of  parameters  would  enable  analysis  of  these  systems, 
and  we  hope  other  researchers  will  be  interested  to  try  these  examples.  We  set 
the  upper  limit  for  SpaceEx  running  time  as  two  hours,  and  an  experiment  is 
said  to  be  out  of  time  (OOT)  if  we  can  not  get  the  result  after  two  hours.  The 
reason  the  STC  scenario  did  not  produce  results  is  due  to  the  use  of  outputs  as 
invariant  conditions  (i.e.,  y  =  Cx)  with  nondeterministic  dynamics,  which  does 
not  seem  to  be  supported  when  using  STC. 

Next,  we  present  briefly  the  reachability  analysis  of  some  small  and  medium- 
size  benchmarks  (i.e.,  less  than  50  dimensions). 

Motor  control  system.  Figure  3.1  depicts  the  reachable  set  of  the  interested 
states  of  the  motor  control  system.  As  shown  in  the  figure,  the  reachable  set 
does  not  reach  to  the  unsafe  region.  Thus,  we  can  conclude  that  the  system 
is  safe  in  the  bounded  time  [0,20s].  A  stronger  conclusion  about  the  safety  of 
the  motor  control  system  may  be  given  by  considering  unbounded  time  safety 
verification. 

Building  model.  Figure  3.2  depicts  the  reachable  set  of  the  this  state  of 
the  building  model.  As  can  be  seen  from  the  figure,  the  reachable  states  of 
the  output  do  not  intersect  the  unsafe  region.  Thus,  we  can  conclude  that  the 
system  is  safe  in  the  bounded  time  [0,20s].  Similar  to  the  above  motor  control 
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Figure  3.1:  Reachable  set  of  inter¬ 
ested  outputs  of  the  motor  control 
system  (in  [0,20s])  and  its  corre¬ 
sponding  unsafe  region  (the  red  re¬ 
gion).  The  reachable  set  of  inter¬ 
ested  outputs  do  not  reach  the  un¬ 
safe  region,  thus  the  system  is  safe 
(in  a  bounded  time  interval  [0, 20s]) . 


Figure  3.2:  Reachable  set  of  inter¬ 
ested  output  of  the  building  model 
system  (in  [0,20s])  and  its  corre¬ 
sponding  unsafe  region  (the  region 
above  the  red  line).  The  reachable 
set  of  interested  output  do  not  reach 
the  unsafe  region,  thus  the  system 
is  safe  (in  a  bounded  time  interval 
[0,20s]). 


system,  a  stronger  conclusion  about  the  safety  of  the  building  model  may  be 
given  by  considering  unbounded  time  safety  verification. 

4  Outlook 

Overall,  we  present  in  this  paper  a  set  benchmarks  for  purely  continuous  linear 
systems  (i.e.,  LTI  systems),  modeled  as  single-mode  hybrid  automata  in  the 
SpaceEx  model  format.  The  benchmarks  range  in  dimensionality  from  tens  to 
thousands  of  dimensions,  and  come  from  many  different  domains.  The  continu¬ 
ous  and  hybrid  verification  community  may  use  these  benchmarks  for  comparing 
methods  and  tools,  especially  with  respect  to  continuous  post  operator  bench¬ 
marking  for  systems  with  a  high  number  of  dimensions.  In  ongoing  and  future 
work,  we  intend  to  introduce  additional  high-dimensional  benchmarks  with  both 
piecewise  affine  dynamics  and  continuous  dynamics  including  ones  originally  en¬ 
coded  as  differential  algebraic  equations  (DAEs),  and  are  also  investigating  for¬ 
malization  of  order-reduction  methods  as  sound  abstractions  using  approximate 
bisimulation  relations  [21] . 
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Abstract 

This  benchmark  suite  consists  of  a  number  of  examples  of  autonomous  multi-agent  sys¬ 
tems  where  the  agent  number  ranges  from  two  to  ten.  The  benchmarks  are  derived  from 
the  field  of  position-based  formation  control  in  autonomous  robotics  and  vehicles.  Their 
models  are  given  as  network  of  hybrid  automata  in  the  SpaceEx  XML  model  format  and 
can  be  transformed  to  other  verification  tools  model  formats  using  HyST,  a  model  trans¬ 
formation  tool.  Safety  of  a  small  benchmark  with  two  agents  is  analyzed  using  SpaceEx. 
Category:  academic  Difficulty:  low  through  challenge 

1  Context  and  Origins 

Intelligent  autonomous  systems  have  been  a  “hot”  research  topic  for  many  years  because  of  its 
rigorous  application  domains  such  as  robotics,  unmanned  aerial  vehicles  (UAV),  autonomous 
cars  and  sensors  networks.  The  challenges  in  modeling,  analysis,  design  and  testing  a  such 
intelligent  system  have  attracted  researchers  from  different  disciplines  such  as  biology,  computer, 
communication  and  control.  In  an  early  step,  the  intelligent  behavior  called  “flocking  behavior” 
of  a  group  of  animals  such  as  bird,  insect  and  fish  has  been  investigated  deeply  over  decades 
in  the  field  of  biology  [1],  The  behavior  has  been  first  modeled  and  simulated  using  computer 
in  [2] .  This  work  has  inspired  a  new  field  of  modeling,  control  and  design  for  autonomous  systems 
which  is  now  considerably  an  important  topic  for  the  next  generation  of  modern  technology. 

Consensus  and  formation  controls  are  two  fundamental  problems  in  designing  an  autonomous 
system  that  perform  an  intelligent  behavior.  Control  scientists  have  proposed  numerous  proto¬ 
cols  over  last  decades  to  drive  the  system  to  achieve  some  control  objectives  [3-9].  Generally, 
to  perform  a  specific  task,  the  agents  need  to  exchange  their  information  and  cooperate  with 
each  other  over  communication  channel.  The  communication  topology  of  an  autonomous  sys¬ 
tem  describes  in  detail  how  the  information  flow  in  the  system.  The  communication  topology 
can  be  static,  i.e.  does  not  change  over  times,  or  dynamics,  i.e.  may  change  over  times.  It 
can  also  be  directed,  i.e.  information  flows  in  one  direction  over  a  connection  between  two 
agents,  or  undirected ,  i.e.  the  information  flows  in  both  directions  over  a  connection  between 
two  agents.  The  communication  topology  expresses  the  sensing  and  communicating  capaci¬ 
ties  of  the  agents  which  affect  significantly  to  the  stability,  controllability  and  the  convergence 
of  an  autonomous  system.  Graph  theory  has  been  proved  as  an  powerful  tool  to  model  the 
communication  topology  and  analyze  the  controllability  of  autonomous  systems  [10] 
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Formation  control  for  autonomous  systems  [7-9]  is  seeking  control  laws  to  guarantee  that  the 
agents  move  to  pre-determined  positions  while  keeping  the  system  formation  in  some  specific 
shapes  when  moving.  Depending  on  the  sensing  and  communicating  capacities  of  the  agents,  i.e. 
the  communication  topology,  the  formation  control  strategies  can  be  categorized  into  position- 
based,  displacement-based  and  distance-based  approaches  [11].  One  essential  safety  requirement 
for  the  system  is  that  there  is  no  collision  when  the  agents  are  moving.  These  formation  control 
strategies  have  shown  informally  the  ability  of  the  agents  avoiding  collision  via  simulation-based 
testing.  To  guarantee  the  safety  of  the  system,  its  formal  model  need  to  be  given  and  verified 
using  formal  verification  techniques. 

Toward  safety  and  liveness  requirements  of  autonomous  systems,  some  control  algorithms 
have  been  proposed  and  verified  using  formal  verification  techniques  recently  [12, 13].  In  this 
context,  the  formal  model  of  an  autonomous  system  is  given  based  on  discrete  time  intervals  and 
to  guarantee  the  safety  of  the  system,  the  controller  usually  can  perform  some  particular  actions 
to  resolve  the  potential  risks  coming.  The  whole  system  is  modeled  as  a  labeled  transition  system 
and  the  safety  and  liveness  requirements  are  written  in  form  of  linear  temporal  logic  (LTL). 

Inspired  by  above  interesting  works,  in  this  paper,  we  obtain  a  set  of  autonomous  systems 
benchmarks  written  in  SpaceEx  XML  format.  Each  agent  is  modeled  separately  as  a  single 
hybrid  automaton  and  the  whole  system  is  a  network  of  hybrid  automata  which  is  basically  a 
composition  of  all  agents.  Different  from  [12,13],  these  benchmarks  have  continuous  dynamics. 
Therefore,  their  safety  requirements  can  be  verified  using  existing  verification  tools  that  support 
verifying  continuous  dynamics  [14-17].  In  addition,  when  the  number  of  agents  increases,  the 
benchmark  models  become  larger  that  makes  them  harder  to  be  verified.  Thus,  our  benchmark 
suite  is  also  useful  for  testing  the  scalability  of  verification  tools. 

The  rest  of  the  paper  is  organized  as  follows:  Section  2  presents  the  description  of  an 
autonomous  system  including  the  communication  topology,  the  motion  dynamics  of  the  agents 
and  the  position-based  formation  control  strategies.  Section  3  gives  the  safety  analysis  of  some 
small  autonomous  systems  using  SpaceEx.  Section  4  discusses  some  interesting  issues  for  the 
future  work  and  concludes  the  paper. 


2  System  descriptions 

2.1  Communication  topology 

Directed/undirected  graphs  are  powerful  tool  for  modeling  the  interaction  between  agents  in  an 
autonomous  system.  In  this  benchmark  suite,  the  communication  topologies  of  all  autonomous 
systems  are  modeled  using  directed  graphs.  A  digraph  (directed  graph)  defined  by  a  tuple 
(V,£),  where  V  is  a  finite  non-empty  set  of  vertices  and  £  £  V2  is  a  set  of  ordered  pairs  of 
vertices,  called  edges.  It  can  be  understood  that  vertice  Vi  £  V  represents  for  the  ith  agent  an 
autonomous  system  and  ordered  edge  (i,j)  represents  for  the  interaction  between  the  agent  i 
and  the  agent  j  where  the  information  flows  from  i  to  j,  i.e.  agent  j  receives  the  information 
from  agent  i.  To  model  how  much  information  flows  in  communication,  we  use  a  weighted 
digraph  which  can  be  defined  by  an  adjacency  matrix  A  =  [atj]nxn,  where  an  =  0,  atj  >  0 
if  (j,i)  6  £  and  n  =  |V|  is  the  number  of  agents  in  the  system.  Figure  2.1  illustrates  an 
example  of  communication  topology  of  an  autonomous  system  with  six  agents  [18].  From  the 
communication  topology,  it  can  be  seen  that  one  agent  only  can  collect  some  information  from 
its  neighbors,  not  from  all  other  agents. _ 
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Figure  2.1:  An  example  of  communication  topology  using  a  weighted  digraph. 


y 


•  ( xhi,yhi ) 


x 


Figure  2.2:  Non-holonomic  differential  driven  mobile  robot. 


A  communication  topology  can  be  static,  as  in  the  case  of  the  example,  or  dynamic,  i.e.  the 
connections  between  agents  can  be  varied  over  times.  A  dynamic  communication  topology  may 
be  convenient  to  characterize  naturally  the  interaction  behaviors  of  agents  in  practice  where  the 
sensing  capacity  of  agents  is  limited  in  some  ranges  and  hence,  it  can  not  recognize  the  other 
agents  outside  of  its  sensing  range.  However,  the  dynamic  communication  topology  increases 
the  difficulty  in  designing  the  control  law  to  guarantee  autonomous  systems  to  perform  the 
intelligent  flocking  behavior.  In  this  paper,  the  benchmarks  can  be  categorized  into  static  or 
dynamic  communication  topology. 

We  have  briefly  introduced  modeling  interaction  between  agents  in  an  autonomous  system 
using  directed  graph.  Next,  we  give  the  dynamics  of  the  agents  and  the  formation  control  rules 
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2.2  Motion  dynamic  and  formation  control 


In  this  paper,  we  consider  the  formation  control  for  multiple  mobile  robots  in  a  2-dimensional 
plan  in  which  the  equations  of  motion  of  a  non-holonomic  mobile  robot  depicted  in  Figure  2.2 
are  given  by 

Xi  =  Vi  cos(0j), 

Vi  =  Vising), 

9i  =  Wi,  (2-1) 

rriiVi  =  fi, 

Ji&i  =  Tj , 

where  ( Xi,yi )  is  the  Cartesian  position  of  the  robot  centre,  8i  is  the  orientation,  Vi  is  the  linear 
velocity,  ivt  is  the  angular  velocity,  rrii  is  the  mass,  Ji  is  the  mass  moment  of  inertia,  ft  is  the 
force,  and  Tj  is  the  torque  applied  to  the  robot. 

Since  Equation  2.1  contains  the  nonlinear  functions  cos (Oi)  and  sin(0j),  the  robot  dynamic 
is  nonlinear  and  thus,  we  cannot  model  and  analyze  the  system  using  SpaceEx.  Fortunately,  we 
can  avoid  the  non-holonomic  constraint  and  obtain  a  linear  model  for  the  system  by  introducing 
intermediate  position  variables  ( Xhi,yhi )  as  follows  [18]. 


%hi 

Vhi 


cos  {Oi) 
sin(0j) 


(2.2) 


We  can  see  that  (x hi,  Vhi)  is  a  position  off  the  wheel  axis  of  the  ith  robot  by  a  distance  di. 
Now,  if  we  let 


fi 

COS(0*) 

-^sin(0i) 

-1 

vXi  +  ViiOi  sin  (Oi)  +  diUif  cos  (Oi) 

n 

_Tsin^) 

-  cos  (Oi) 

vyi  —  ViiOi  cos  (Oi)  +  diUjf  sm(Oi) 

Then  we  can  obtain  the  new  linear  equations  of  motion  for  each  robot  as  a  double-integrator 
system: 


%hi  ^xii 
Vxi  —  bxii 

Vhi  —  ^ yii 

%r  ~  byt. 


(2.3) 
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The  control  objective  is  to  drive  the  mobile  robots  from  their  initial  location  (x(fu.  y Jk)  to  pre¬ 
defined  destinations  (x^- ,  yfLi )  while  preserving  the  formation  of  the  system  during  the  transition, 
e.g.,  a  square  formation  for  4-robots  team,  a  triangle  formation  for  3-robots  team.  Assume  we 
have  the  communication  topology  of  the  system  defined  by  adjacent  matrix  A  =  [o.tj\nxn,  where 
n  is  the  number  of  agents  in  the  system,  the  position-based  formation  control  law  for  the  system 
is  designed  as  follows  [18]. 

n  n 

bxi  (x hi  X hi )  '^Ix^X-xX'hi  ^  '  ^i'j  \  (.Xhi  Xfof)  (x  hj  Xfoj)\  ^  ^  7 x^ij  hi  X fij ) 

2=1  2=1 
n  n 

byi  =  - av{yhi  -  yti)  -  lyayyhi  -  ^  aij[(yhi  -  yt)  -  {yhj  -  yij)\  -^TyaijiVhi  ~  Vhj ) 

2=1  2=1 

(2.4) 

where  a*  >  0  and  7*  >  0. 

The  first  two  terms  of  the  control  law  are  responsible  for  driving  each  robot  to  its  destina¬ 
tion  (goal  seeking)  while  the  last  two  terms  of  the  control  law  are  to  preserve  the  formation 
between  robots  (formation  keeping).  In  term  of  verification,  there  are  both  liveness  and  safety 
properties  need  to  be  verified.  The  liveness  property  relates  to  goal  seeking  objective  as  we 
need  to  guarantee  that  each  robot  finally  reach  its  destination.  The  safety  property  concerns 
the  formation  keeping  problem  as  it  is  required  there  is  no  collision  when  robots  are  moving. 

With  above  formation  control  law,  we  can  derive  the  closed-loop  dynamic  equation  for  the 
system.  Let  xei  =  xhi  -  xdhi,  yei  =  yhi  -  y^,  xe  =  [xei,  ...,xen]T  and  ye  =  [ye  1,  ...,yen]T,  the 
closed-loop  dynamic  of  the  system  can  be  written  by 


Xe 

Onxn 

In 

xe 

Xe 

—  (L  +  axIn ) 

Tx(L  -\-  axIn) 

Xe 

Ve 

Onxn 

In 

Ve 

Ve 

—  (L  +  ayIn) 

-7 y(L  +  ayIn) 

Ve 

where  0raXn  is  n-dimensional  square  zero  matrix,  In  is  n-dimensional  identity  matrix  and  L  = 
[kj\nxn  in  which  lu  =  YjjjH  aij  ancl  hj  =  - a,ij ,  where  i  7^  j. 

We  have  already  described  the  communication  topology,  the  system  dynamics  and  formation 
control  law.  Next,  we  formally  define  the  safety  property  for  the  system. 

3  Safety  property 

Informally,  the  system  is  safe  if  there  is  no  collision  when  the  robots  move  to  their  destination.  In 
other  word,  the  distance  between  two  arbitrary  robots  (i.e. ,  the  distance  between  their  centers) 
need  to  be  larger  than  the  diameter  of  the  robots.  Recall  that  the  robots  shapes  are  circles  and 
their  sizes  are  identical.  The  distance  between  the  ith  and  jth  robots  is 

dij  =  yjixi-xj)2  +  (yi-yj)2. 

Let  V  be  the  diameter  of  the  robot.  The  safety  property  S  of  the  system  can  be  defined 
formally  as  follows 

_ S  :  VL  j,  i  ±  j,  t  >  0,  >  V. _ (3.1) 
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The  dual  unsafe  specification  U  for  two  arbitrary  robots  can  be  defined  by  the  following 
circle. 

U  :  {xj  —  Xj)2  +  (ui  —  yj)2  <  V2 .  (3.2) 

From  Equation  2.2,  we  have: 

{%hi  hj )  T  dj )  ( Xi  Xj )  ^  ( X-hi  X hj )  T  T  dj )  ^  ^ 

(2/w  -  Vhj)  -  {di  +  dj)  <  (yi  -  yj)  <  (yhi  -  +  (d*  +  dj) 

The  above  inequality  shows  that  we  can  compute  the  reachable  sets  of  {x^  —  Xj)  and  (y^  —  Vj) 
by  bloating  the  reachable  sets  of  ( Xhi  —  Xhj )  and  ( yhi  —  yh.j )  by  ( di  +  dj ).  Then,  using  the  bloated 
reachable  sets,  we  can  check  whether  they  violate  the  safety  property  (i.e.,  whether  the  reachable 
sets  reach  the  corresponding  unsafe  region  defined  in  Equation  3.2). 

We  have  formally  defined  the  safety  property  of  the  system  and  described  briefly  how  to 
check  the  safety  of  the  system.  Next,  we  discuss  how  to  model  the  distributed  autonomous 
system  using  hybrid  automata. 


4  System  modeling 

There  are  three  approaches  for  modeling  an  autonomous  system  using  hybrid  automata  frame¬ 
work.  The  first  approach  is  that  we  can  model  the  system  using  decentralized  style  in  which  each 
agent  as  a  hybrid  automata  network  composed  by  dynamic  component  describing  the  dynamic  of 
the  agent  as  defined  in  Equation  2.3  and  controller  component  describing  the  distributed  forma¬ 
tion  control  law  in  Equation  2.4.  Since  the  communication  topology  of  the  autonomous  system 
may  change,  the  controller  component  may  switch  its  operation  between  different  modes.  The 
whole  system  will  be  a  network  of  hybrid  automata  composing  n  agent’s  models.  In  the  second 
approach,  we  can  model  the  system  using  centralized  style  in  which  each  agent  is  a  single-mode 
hybrid  automaton  describing  the  dynamic  of  the  agent,  the  control  law  given  in  Equation  2.4 
is  modeled  as  a  centralized  coordinator  which  is  a  hybrid  automata  containing  one  or  multiple 
modes.  Last  but  not  least,  we  can  also  model  the  system  as  one  single  automaton  describing 
the  closed-loop  dynamic  defined  by  Equation  2.5. 

The  first  two  modeling  approaches  have  two  advantages.  First,  they  describe  intuitively  the 
hierarchical  architecture  of  the  system  in  which  each  agent  is  a  separate  entity.  The  obtained 
model  in  the  first  modeling  approach  illustrates  the  decentralized  control  strategy  in  autonomous 
systems  where  the  control  signal  is  computed  at  agent  side.  In  contrast  to  decentralized  control 
strategy,  the  second  approach  describes  the  centralized  control  strategy  where  the  coordinator 
collects  the  information  of  all  agents  and  computes  the  control  signals  before  sending  them  to 
the  agents.  Second,  since  the  first  two  modeling  approaches  separate  the  agent’s  dynamic  and 
the  control  law,  they  are  convenient  for  changing  the  dynamics  of  the  agents  and  they  also 
allow  modeling  the  switching  happen  between  different  dynamics  of  one  agent.  In  addition,  it 
is  easy  to  model  and  verify  the  system  under  a  complex  hybrid  control  law  when  the  controller 
switches  between  different  modes  along  with  communication  topology  changes.  While  the  first 
two  modeling  approaches  are  convenient  for  modeling  complex  autonomous  systems,  the  third 
approach  is  useful  for  finding  an  abstraction  for  the  whole  system  that  allows  us  to  verify  a  very 
large  autonomous  system  using  order-reduction  abstraction  method  [19].  In  this  benchmark 
suite,  we  use  the  first  and  the  second  approaches  to  model  distributed  autonomous  systems. 
Examples  of  these  modeling  approaches  arc  depicted  in  Figure  4.1  and  Figure  4.2. _ 
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Figure  4.1:  Decentralized- style  approach  for  modeling  autonomous  systems  using  hybrid  au¬ 
tomata  network. 


5  Reachability  analysis 

The  benchmark  suite  including  12  benchmarks  (MAS2  —  MASIO3)  is  presented  in  Table  5.1. 
In  this  paper,  we  present  briefly  the  safety  analysis  of  the  benchmark  MAS 2  with  two  agents. 
The  communication  topology  of  MAS 2  shows  that  the  robot  2  receive  the  information  from 
the  robot  1.  The  initial  intermediate  positions  of  two  robots  are  (2;°-,,  y^)  and  ( x °2,  y°2)  where 
(0  <  xqm  <  0.2,0  <  <  0.1)  and  (0  <  x^2  <  0.2,  0.9  <  y°2  =  1).  Assume  that  the  distances 

between  the  intermediate  positions  and  their  corresponding  robot  centers  are  d\  =  d2  =  l  =  0.1. 
The  robots  are  controlled  to  go  to  their  intermediate  destinations  (xfx  =  3 =  3)  and 
(xh2  =  4,  Vh2  =  4)  while  keeping  their  intermediate  distance  dh  >  1  as  moving.  The  system  is 
safe  if  the  distance  between  two  robots,  i.e.,  between  the  centers  of  two  robots,  is  always  larger 
than  a  threshold  dmin  =  0.5.  We  need  to  ensure  that  this  threshold  is  larger  than  the  size  of 
the  robots  (i.e.,  the  diameter  of  the  robots).  The  parameters  for  the  distributed  control  law  in 
Equation  2.4  are  chosen  as  follows:  ax  =  2ay  =  2,  ■yx  =  2jy  =  1. 

Figure  5.1  describes  the  trajectories  of  the  two  robots.  The  figure  shows  that  two  robots 
finally  reach  their  destinations. 

Recall  that  (xhi,yhi)  and  (Xh2,  y/12)  are  not  the  centers  of  the  robots  as  given  in  Equation  2.2. 
To  verify  the  system  safety,  let  disx  =  X2  —  Xi,  disy  =  2/2  —  2/i ,  disxh  =  Xh.2  —  Xhi,  disyh  = 
Uh2  —  y hi,  the  unsafe  region  of  the  system  can  be  defined  by  the  following  circle. 

\disx\2  +  \disv\2  <  d2min 

If  the  unsafe  region  can  not  be  reached,  then  two  robots  are  always  far  away  from  each  other 
at  a  distance  d  >  dmin  and  then,  we  can  conclude  that  the  system  is  safe.  From  Equation  3.3, 
the  reachable  sets  of  disx  and  disy  can  be  derived  by  bloating  the  reachable  sets  of  disxh  and 
diSyh  using  the  following  constraints. 

Xh2  ~  Xhl  -  21  <  X2  -  X!  <  xh2  -  xhi  +  21 
yh2  -  yhi  ~  2Z  <  y2  -  2/i  <  Vh2  ~  Vhi  +  2 1 
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Figure  4.2:  Centralized- style  approach  for  modeling  autonomous  systems  using  hybrid  automata 
network. 


Figure  5.2  and  Figure  5.3  illustrate  the  reachable  set  of  disxh  and  disyh  over  times  and  Fig¬ 
ure  5.4  describes  the  reachable  set  of  ( disx ,  disy)  (the  green  polygon)  which  is  bloated  from  the 
reachable  set  of  ( disxh ,  disyh)  (the  green  polygon).  The  later  figure  shows  that  ( disx ,  disy)  does 
not  reach  the  unsafe  region  for  all  times  when  the  robots  move  to  their  destinations.  Thus, 
we  can  conclude  that  the  system  is  safe.  In  addition,  we  can  see  that  the  formation  control 
law  actually  works  since  it  drives  the  robots  to  their  destinations  and  preserve  the  formation 
of  the  robots  when  they  are  moving  (i.e. ,  the  intermediate  distance  between  the  robots  finally 
converge  to  dh  =  V%- 

It  is  worth  noticing  that  the  control  parameters  {ax,  ay,  jx,  7y}  assigned  in  Equation  2.4 
affects  significantly  the  performance  of  the  system.  As  analyzed  in  [18],  there  exists  conditions 
for  the  control  parameters  and  the  communication  topology  to  guarantee  that  the  robots  can 
finally  reach  their  destination  while  preserving  their  formation.  An  appropriate  choices  of  the 
control  parameters  can  be  given  from  these  conditions.  In  addition,  the  initial  condition  and 
the  destination  requirements  (i.e.,  the  destination  positions  of  the  robots)  also  affects  the  safety 
property  of  the  system.  For  example,  if  the  destination  requirements  conflict  with  the  formation, 
the  collision  may  occur. _ 
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Figure  5.1:  Trajectories  of  the  two  robots. 


Figure  5.2:  Reachable  set  of  disxh  =  Xh2 
Xhi  over  times 

6  Outlook 


Figure  5.3:  Reachable  set  of  disyh  =  Uh2 
yhi  over  times 


Overall,  we  present  in  this  paper  a  set  benchmarks  for  distributed  autonomous  systems,  modeled 
as  network  of  hybrid  automata  in  the  SpaceEx  model  format.  The  number  of  the  agents  range 
from  two  to  ten.  The  position-based  formation  control  has  been  successfully  verified  in  a 
benchmark  with  two  agents.  There  are  two  important  issues  should  he  considered  in  future 
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Figure  5.4:  Reachable  set  of  ( disx,disy )  (the  green  polygon),  (disxh,  disyh)  (the  blue  polygon) 
and  the  unsafe  region  (inside  the  red  circle). 

work.  First,  it  is  challenging  to  model  and  verify  the  safety  and  livenesss  properties  of  the 
distributed  autonomous  systems  controlled  by  complex  nonlinear  formation  control  laws  to 
avoid  collision  and  obstacles.  We  can  take  advantages  of  verification  tools  supporting  nonlinear 
hybrid  systems  such  as  Flow*  [17]  and  C2E2  [20]  in  this  case.  Second,  it  would  be  useful  for 
testing  verification  tools  if  we  can  generate  automatically  distributed  autonomous  systems  with 
arbitrary  large  number  of  agents.  We  are  going  to  implement  this  feature  as  an  automatic 
generator  in  Hyst  [21]. 
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Abstract  Order-reduction  is  a  standard  automated  approximation  technique  for 
computer-aided  design,  analysis,  and  simulation  of  many  classes  of  systems,  from 
circuits  to  buildings.  To  be  used  as  a  sound  abstraction  for  formal  verification,  a 
measure  of  the  similarity  of  behavior  must  be  formalized  and  computed,  which  we 
develop  in  a  computational  way  for  a  class  of  asymptotic  stable  linear  systems  as 
the  main  contributions  of  this  paper.  We  have  implemented  the  order-reduction  as 
a  sound  abstraction  process  through  a  source-to-source  model  transformation  in  the 
HyST  tool  and  use  SpaceEx  to  compute  sets  of  reachable  states  to  verify  properties 
of  the  full-order  system  through  analysis  of  the  reduced-order  system.  Our  experi¬ 
mental  results  suggest  systems  with  thousand  of  state  variables  can  be  reduced  to 
systems  with  tens  of  state  variables  such  that  the  order-reduction  overapproximation 
error  is  small  enough  to  prove  or  disprove  safety  properties  of  interest  using  current 
reachability  analysis  tools.  Our  results  illustrate  this  approach  is  effective  in  tackling 
the  state-space  explosion  problem  for  verification  of  high-dimensional  linear  systems. 

Keywords  Abstraction;  model  reduction;  order  reduction;  verification;  reachability 
analysis 


1  Introduction 

The  state-space  explosion  problem  is  a  fundamental  challenge  in  model  checking  and 
automated  formal  verification  that  has  received  significant  attention  from  the  veri¬ 
fication  community.  Among  many  solutions,  abstractions  based  on  the  concepts  of 
exact  and  approximate  simulation  and  bisimulation  relations  have  proved  to  be  effec¬ 
tive  approaches  for  obtaining  smaller  state  spaces  by  abstracting  away  information 
that  is  not  needed  in  the  verification  process.  Such  abstractions  have  been  applied 
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broadly  to  simplify  the  controller  synthesis  and  safety  verification  process  for  com¬ 
plex  systems.  Exact  bisimulation  relation-based  abstractions  for  safety  verification 
and  controller  synthesis  have  been  investigated  widely  in  the  last  decade  [Pappas 
(2003);  van  der  Schaft  (2004);  Tanner  and  Pappas  (2003);  Tabuada  and  Pappas 
(2004)].  In  this  context,  the  outputs  of  the  abstract  system  capture  exactly  the  out¬ 
puts  of  the  original  system.  As  pointed  out  in  [Girard  et  al.  (2008);  Girard  and 
Pappas  (2007)],  the  term  “exact”  is  not  adequate  when  dealing  with  continuous  and 
hybrid  systems  observed  over  real  numbers  since  there  may  be  numerical  errors  in 
observation,  noise,  and  other  imperfections.  To  obtain  an  abstraction  that  guaran¬ 
tees  more  robust  relationship  between  systems,  approximate  bisimulation  has  been 
proposed  and  studied  extensively  in  recent  years  [Girard  et  al.  (2008);  Girard  and 
Pappas  (2007);  Julius  (2006);  Girard  et  al.  (2006);  Islam  et  al.  (2015)].  The  main  ad¬ 
vantage  of  this  approach  is  that  they  allow  a  bounded  precision  5  which  describes  how 
“far  off”  the  executions  of  the  abstraction  may  be  from  those  of  the  original  system. 
Then,  verifying  whether  the  executions  of  the  original  system  reach  an  unsafe  region 
U  can  be  reduced  to  verify  whether  the  executions  of  the  abstraction  (with  much 
lower  dimension)  reach  the  (^-neighborhood  of  the  unsafe  region  U.  Thus,  finding  an 
efficient  way  of  computing  a  tight  bound  on  the  precision  becomes  an  essential  task 
for  this  approach.  The  proposed  method  shows  a  great  benefit  when  it  can  deal  both 
stable  and  unstable  systems.  Balanced  truncation  model  reduction,  a  well-known 
technique  in  control  [Antoulas  et  al.  (2001)],  has  recently  been  applied  to  obtain  ab¬ 
stractions  for  formal  verification  of  continuous  and  hybrid  systems  [Han  and  Krogh 
(2004);  Han  (2005)].  The  bounded  precision  between  a  system  and  its  abstraction, 
which  is  also  essential  in  this  framework,  is  determined  using  simulation. 

Inspired  by  the  results  reported  in  [Girard  and  Pappas  (2007);  Han  and  Krogh 
(2004)],  in  this  paper,  we  discuss  and  improve  the  computation  frameworks  in  existing 
techniques  to  make  them  more  applicable  to  practical  higli-dimensional  systems. 
The  main  contribution  of  our  work  is  improving  the  way  of  computing  and  using 
the  precision  5.  Our  method  is  shown  to  be  efficient,  robust  and  scalable  compared 
with  existing  approaches  over  a  set  of  practical  benchmarks  (several  to  a  thousand 
dimensions).  We  implement  the  method  as  a  model  transformation  pass  within  the 
HyST  model  transformation  tool  [Bak  et  al.  (2015)]  which  enables  us  to  easily  apply 
the  techniques  and  compare  results. 

The  remainder  of  the  paper  is  organized  as  follows.  Section  2  gives  essential 
definitions  used  throughout  the  paper.  Section  3  presents  methods  to  find  output 
abstractions  of  the  linear  time  invariant  (LTI)  systems  using  the  balanced  trunca¬ 
tion  model  reduction  method.  Section  4  discusses  how  to  verify  safety  properties  for 
a  full-order  LTI  system  using  its  output  abstraction.  Section  5  describes  our  imple¬ 
mentation  of  the  method  in  a  prototype  tool,  and  presents  a  number  of  examples  to 
illustrate  and  evaluate  the  benefits  of  our  method. 


2  Preliminaries 

Consider  two  asymptotically  stable  LTI  systems: 
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where  x(t)  G  1",  xr(t)  E  Rfc  are  the  system  states;  y(t)  E  Rp,  yr(t)  E  Rp  are 
the  system  outputs;  u(t)  is  the  control  input;  A  E  Rnxn,  Ar  E  Rfcxfc,  B  E  Rnxm, 
Br  E  Rfcxm  and  C  E  Rpxn,  Cr  E  Rpxfc  are  system  matrices.  The  initial  set  of  states 
of  Mn  and  Mk  are  respectively  denoted  by  X0(Mn )  C  R™,  X0(Mk)  C  Rfc  and  the 
set  of  control  inputs  is  denoted  by  U.  We  write  initial  conditions  as  a;(0)  G  X0(Mn ), 
ov(0)  G  X0(Mfc)  and  the  control  input  as  u(s)  G  U, Vs  G  [0,t].  In  this  paper,  we 
assume  that  Mn  is  observable  and  controllable. 

Definition  1  Output  Abstraction.  M &  is  called  a  k-dimensional  output  abstraction 
of  Mn  with  precision  5  =  [<5i ,  <52 ,  - .  - ,  Sp]T ,  denoted  as  M^,  where  each  Si  is  a  finite 
positive  real  if,  for  Vx(0)  G  X0(Mn)  and  u  E  U,  3a;T.(0)  G  Xq(MU)  such  that,  Vt  >  0, 

1 1 y1 (t)  —  y\. (t)  1 1  <  Si,  1  <  i  <  p  where  yl(t)  is  the  ith  component  of  the  output  y  at 
time  t,  and  ||-||  denotes  the  Euclidean  norm. 

Informally,  the  output  abstraction  behaviors ,  which  are  defined  as  the  trajectories  of 
the  output  over  real  time  intervals,  will  approximate  within  5  the  behaviors  of  the 
full-order  system  Mn  for  all  time. 

Definition  2  Output  Reach  Set  [Han  and  Krogh  (2004)].  The  output  reach  sets  at 
a  time  instant  t  and  over  a  time  interval  [0,t/],  t/  >  0  of  Mn  respectively  are: 

Rt(Mn)  =  {y(t,u,x(0))\y(t,u,x(0))  =  CeAtx(0)  +  f  Ceyt(-t_T'i?u(r)dr}, 

Jo 

R[o,tf]{Mn)  =  M  Rt{Mn),  where  s(0)  G  X0{Mn),  u(r)  E  U,Vt  E  [0, t] . 

Definition  3  Safety  Specification.  A  safety  specification  S(M„ )  of  an  LTI  system 
Mn  formalizes  the  safety  requirements  for  the  output  y  of  M„,  and  is  a  predicate  over 
the  output  y  of  Mn.  Formally,  S(Mn)  C  RP.  The  dual  unsafe  specification  U(Mn)  of 
the  system  is  also  a  predicate  over  the  system  output,  i.e.  U(Mn )  C  Rp. 

Definition  4  Safety  Verification.  The  time-bounded  safety  verification  problem  is  to 
verify  whether  the  system  Mn  satisfies  a  safety  specification  S(Mn)  over  an  interval 
of  time  [0,  tf\  (tf  is  finite  and  positive),  which  is  described  formally  in  terms  of  the 
output  reach  set  as: 

R[0,tf](Mn)  n  ~>S(Mn)  =  0  Mn  1=  S(Mn), 

R{0,tf]{Mn)  n  ^S(Mn)  S(Mn). 

If  Mn  satisfies  S(Mn ),  then  it  is  safe  and  we  write  Mn  t=  S(Mn).  If  Mn  does  not 
satisfy  S(Mn),  then  it  is  unsafe  and  we  write  Mn  ¥  S(Mn).  The  notation  denotes 
the  logical  negation  which  corresponds  to  set  complement  in  the  formulas  above. 

Definition  5  Safety  Specification  Transformation.  The  safety  specification  trans¬ 
formation  is  the  process  of  finding  the  corresponding  safety  (or  dually,  unsafe) 
specification  for  the  output  abstraction  denoted  by  S'(M^)  G  Rp  (and  dually 
E  Rp)  from  the  safety  specification  S(AIn)  of  the  full-order  system  Mn  to 
guarantee  the  safety  relation  defined  by: 

R[o,tf]{Mk)  n  -S(Mf)  =  0  =>  Mn  h  S(Mn), 

^[0lt/]  {Mi)  n  U{Ml)  ^  0  =►  Mn  *  S(Mn). 

The  main  objective  of  this  paper  is  to  perform  the  safety  verification  for  high¬ 
dimensional  LTI  systems  using  output  abstraction  and  its  transformed  safety  speci¬ 
fication  to  reduce  the  computational  complexity. 
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3  Output  Abstractions  from  Balanced  Truncation  Reduction 


The  balanced  truncation  model  reduction  (BTMR)  [Moore  (1981)]  is  a  well-known 
method  in  control  which  is  used  to  find  a  reduced  model  M t  for  an  asymptotically 
stable  large  scale  linear  system  Mn  so  that  its  output  trajectory  captures  the  output 
trajectory  of  the  original  system  within  a  estimated  bounded  error  under  an  identity 
control  input.  The  first  step  in  BTMR  is  to  implement  a  balanced  transformation 
x(t)  =  Hx(t),  H  £  Rnxn  to  transform  Mn  to  an  equivalent  balanced  system  Mn. 
The  fc-order  reduced  model  which  is  also  asymptotically  stable  is  then  obtained 
by  selecting  the  first  k  states  in  the  state  vector  of  Mn  and  truncating  the  other  n  —  k 
states  (i.e.  xr  =  Sx(t)  =  SHx(t ),  S  =  ( Ikxk  0 fcx(n-fc)))-  The  (n  +  fc)-dimensional 
(asymptotically  stable)  augmented  system  Mn+k  is  defined  by: 

-x  =  Ax+Bu=  (o£)*+(!r)  «> 

y  =  Cx  =  ( C  —Cr)  x. 

^  ;  A,  B  and  C  are  the  matrices  of  balanced  system  Mn. 

To  derive  an  output  abstraction  M £  from  the  system  Mn,  we  first  use  BTMR 
to  obtain  the  reduced  model  M and  then  determine  the  precision  5  which  relates 
not  only  to  the  control  input  u(t)  but  also  to  the  initial  condition  a;(0).  Determining 
the  precision  5  is  equivalent  to  determining  the  bounds  of  the  augmented  system’s 
individual  outputs.  Let  ei(f)  =  CeAtx( 0)  be  the  zero  input  response  and  e2 (t)  = 
CeA^~T^ Bu(r)dr  be  the  zero  state  response.  It  is  easy  to  see  that  ||  yl(t)  —  ylr{t)  ||  = 
||yI(£)||  <  1 1 e ^ (£)  1 1  +  1 1 (t)  1 1  =  <5i;  where  hz(t)  denotes  the  ith  element  of  vector  h(t). 

The  following  theorems  obtain  the  theoretical  bounds  for  ei(f)  and  e2  (f).  The 
sum  of  these  two  bounds  gives  us  the  precision  5. 

Theorem  1  Let  a:(0)  =  {Hx{ 0)  SHx{ 0))T,  then  the  zero  input  response  e\{t)  of 
the  asymptotically  stable  augmented  system  Mn+k  satisfies  the  following  inequality 
for  all  t  £  R>o: 

1 1  ei  (t)  1 1  <  ||A||  'Supl(o)£y0  ||i(0)||  ,  1  <i<P, 
where  M>o  is  the  set  of  non-negative  real  numbers,  Ci  is  the  row  i  of  the  matrix  C . 

The  proof  of  Theorem  1  is  given  in  Appendix  7.1. 

Theorem  2  Let  a;(0)  =  (Hxf  0)  SHx{ 0))T  and  Pq  >  0  is  the  solution  of  the  follow¬ 
ing  optimization  problem: 

Pq  =  min(trace(P))  subject  to 
P  >  0,  ATP  +  PA  <  0,  Cf  Ci  <  P 

where  C-i  is  the  row  i  of  the  matrix  C.  Then,  the  zero  input  response  e±(t)  of  the 
asymptotically  stable  augmented  system  Mn+k  satisfies  the  following  inequality  for 
all  t  £  R>o-' 

||ei(*)fl  <  supl(o)£x0  \/x{0)TPox{0),  1  <i<P, 


where  x  = 
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The  proof  of  Theorem  2  is  given  in  Appendix  7.2. 

Theorem  3  [Han  and  Krogh  (2004);  Obinata  and  Anderson  (2012)].  The  error  e 2 
between  the  full-order  asymptotically  stable  system  Mn  and  its  k-dimensional  reduced 
system  M [  satisfies  the  following  inequality  for  all  t  G  IR>o.' 

ETL 

■  n1(2J“1W'  Halloo  > 

j=k-\- 1 

where  <jj  is  the  jth  Hankel  singular  value  of  the  system  Mn  and  Uu^)!^  =  suptgR>0  ||ti(f)|| 

We  note  that  the  precision  5  can  be  obtained  using  different  methods  all  of  which 
have  their  advantages  and  drawbacks.  In  [Han  and  Krogh  (2004)],  the  authors  pro¬ 
pose  a  simulation-based  approach  to  determine  5.  To  determine  the  bound  of  ei, 
the  authors  simulate  the  full-order  system  and  the  reduced  system  from  each  vertex 
in  a  polyhedral  representation  of  the  initial  set  of  states.  The  advantage  in  using 
this  method  is  that  it  gives  a  very  tight  bound  of  e\  while  the  drawback  is  that  the 
number  of  simulations  required  can  grow  exponentially.  For  example,  if  the  initial 
set  is  a  hypercube  in  100-dimensions,  we  have  to  simulate  the  full-order  system  and 
its  reduced  system  with  2"  =  2100  vertices,  which  is  infeasible  even  if  each  simula¬ 
tion  takes  little  time.  The  bound  of  e2  is  determined  by  integrating  the  norm  of  the 
impulse  response  of  the  augmented  system  via  simulation.  This  method  is  especially 
useful  in  practice  since  it  gives  a  tight  bound  for  e2  with  only  m  simulations,  where 
m  is  the  number  of  inputs.  Alternatively,  without  separately  computing  the  bounds 
of  e±  and  e2,  the  precision  S  can  be  calculated  by  solving  a  set  of  LMI  optimization 
problems  on  sets  of  initial  states  and  inputs  [Girard  and  Pappas  (2007)].  This  ap¬ 
proach  has  advantages  when  dealing  with  small  and  medium-dimensional  systems 
(less  than  50  dimensions)  and  works  for  both  stable  and  unstable  systems.  When  the 
system  dimension  is  large,  the  error  bound  obtained  is  overly  conservative  and  may 
not  be  useful. 

Exploiting  the  advantages  and  overcoming  the  drawbacks  of  existing  approaches 
when  dealing  with  practical  systems  are  the  main  contributions  of  our  approach. 
First,  we  compute  the  bounds  for  e\  and  e2  separately,  making  the  computation 
process  more  robust  and  thus  produce  less  conservative  results  in  comparison  with 
[Girard  and  Pappas  (2007)].  Second,  Theorem  1  and  Theorem  2  are  proposed  to 
overcome  the  drawback  of  simulation-based  method  [Han  and  Krogh  (2004)]  as  de¬ 
scribed  above.  It  should  be  emphasized  that  in  most  circumstances,  our  approach  can 
be  used  to  compute  the  bound  of  e\  and  the  simulation-based  approach  can  be  used 
to  determine  the  bound  of  e2-  This  combination  is  scalable  for  high-dimensional  sys¬ 
tems  while  still  obtaining  a  good  precision  5.  Finally,  the  output  abstraction  defined 
based  on  individual  outputs  (y'(t),ylr(t))  and  precisions  5t  allows  us  to  easily  verify 
the  safety  of  multi-output  systems.  Next,  we  briefly  analyze  the  time  complexity  in 
computing  the  precision  5  to  show  theoretically  the  scalability  of  different  methods. 

We  analyze  the  complexity  of  the  simulation-based  approach  [Han  and  Krogh 
(2004)]  in  terms  of  number  of  simulations.  For  an  n-dimension  system  with  m  inputs 
and  p  outputs,  the  number  of  vertices  in  a  polyhedral  initial  set  (in  the  worst  case)  is 
2".  Therefore,  the  number  of  simulations  needed  to  determine  the  bound  of  ei  is  2". 

For  e2,  the  number  of  simulations  needed  is  m.  Consequently,  the  number  of  simula¬ 
tions  needed  is  0((2n+m)).  In  [Girard  and  Pappas  (2007)],  to  determine  the  precision 
5,  this  method  solves  two  LMIs  and  quadratic  optimization  problems  on  the  sets  of 
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[Girard  and  Pappas  (2007)] 

[Han  and  Krogh  (2004)] 

0(  2n  +  m) 

Theorem  1 

0(nsb) 

Theorem  2 

0((n  +  fc)5'5) 

Table  1:  Time  complexity  of  different  methods  to  compute  the  precision  5. 


initial  state  and  inputs  in  which  the  time  complexity  for  solving  two  LMI  constraints 
using  interior  point  algorithms  can  be  estimated  by  0([(n2  +  n) / 2]2-75  x  215)  [Van- 
denberghe  and  Boyd  (1994);  Nesterov  et  al.  (1994)]  and  the  time  complexity  for 
solving  the  optimization  problem  in  [Girard  and  Pappas  (2007)]  using  interior  point 
algorithms  is  0([(n2  +  n)/2]3).  Overall,  the  time  complexity  of  computing  the  preci¬ 
sion  is  0([(n2  +  n)/2]3)  +  0([(n2-|-n)/2]2-75  x  215),  which  can  be  bounded  by  0(n6). 
In  our  approach,  it  is  easy  to  see  that  Theorem  1  computation  mainly  relates  to 
solving  the  optimization  problem  to  find  supx(-0)gXo  ||x(0)||.  The  time  complexity  for 
solving  this  problem  using  interior  point  algorithms  is  0((n  +  l)3  5).  For  Theorem  2, 
we  first  need  to  solve  the  eigenvalue  problem  (EVP)  subject  to  two  matrix  inequal¬ 
ities  that  has  time  complexity  0([((n  +  fc)2  +  n  +  fc)/ 2]2’75  x  21'5)  if  using  interior 
point  algorithms.  Then,  we  need  to  solve  the  quadratic  optimization  problem  that 
has  time  complexity  0((n+  fc)3)  (using  the  interior  point  algorithm).  Overall,  the 
time  complexity  of  Theorem  2  can  be  bounded  by  0((n+  fc)5  5).  The  computational 
cost  of  Theorem  3  in  our  approach  is  smaller  compared  to  Theorem  1  and  Theo¬ 
rem  2.  Table  1  shows  the  simplified  time  complexity  analysis  of  different  approaches. 
The  computation  time  of  these  approaches  will  be  measured  and  discussed  in  detail 
in  Section  5. 


4  Safety  Verification  with  Output  Abstractions 

It  should  be  emphasized  that  first,  our  approach  is  different  from  [Han  and  Krogh 
(2004)]  in  the  way  of  using  the  precision  <5  where  the  precision  is  used  to  compute  the 
reach  set  of  the  full-order  system  before  using  this  reach  set  to  verify  the  safety  of  the 
system.  In  our  approach,  the  precision  is  used  to  obtain  the  safety  specification  of  the 
output  abstraction  that  satisfies  the  safety  relation  (1).  Then,  we  verify  the  safety 
of  the  output  abstraction  to  conclude  safety  of  the  original  system.  It  is  important 
to  notice  from  (1)  that  it  may  be  the  case  that  we  cannot  conclude  anything  about 
the  safety  of  the  original  system  using  the  output  abstraction.  Second,  the  output 
abstraction  is  different  from  the  approximate  abstraction  [Girard  and  Pappas  (2007)] 
which  depends  on  a  single  precision  S  because  it  is  defined  based  on  individual 
outputs  ylr (t))  and  individual  precisions  ft.  As  a  result,  we  can  get  a  more 

accurate  safety  specification  transformation  based  on  the  individual  precisions.  This 
transformation  is  convenient  for  verifying  safety  in  practical  multi-output  systems 
where  the  magnitudes  of  element  outputs  are  usually  significantly  different  from  each 
other.  Our  safety  transformation  rules  are  addressed  in  the  following. 

S(Mn)  as  Convex  Polytopes.  Assume  that  the  safety  specification  of  the  full-order 
system  is  in  the  following  form: 

S(Mn)  =  {y  e  w\  ry  +  v  <  o,  r  =  [atJ]  e  R9Xp,  *  =  [ft]  e  R?},  (2) 
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Lemma  1  Given  S(Mn)  described  by  (2),  then  S(Mk)  and  U(Mjj.)  defined  as  follows 
guarantee  the  safety  relation  (1). 


S(Msk)  =  {yr  £  RP|  Tyr  +  <  0,  W  =  E  +  A}, 

U(Mk)  =  {yr  £  Rp\  Tyr  +  E  >  0,  E_  =  E  -  A}. 


(3) 


where  A  =  [Ai]  £  M9,  At  =  Y%=  i 
The  proof  is  given  in  Appendix  7.1. 

S(Mn)  as  Ellipsoids.  Assume  that  the  safety  specification  of  the  full-order  system  is 
described  by  an  ellipsoid  with  radius  R  and  centered  at  o  €  Rp  as: 

S(Mn)  =  {y£  Mp|  (y  -  a)TQ(y  - a)  <  R 2,  a£P,  R  >  0,  0  <  Q  £  Rpxp},  (4) 

Since  Q  is  a  symmetric  matrix,  there  exists  an  orthogonal  matrix  E  =  [Zi ,  1%, ..,  lp\ 
=  [7 ij]  £  ILpxp  such  that  ETQE  =  A  =  diag{ Ai,  A2,  ...,AP),  where  \  (>  0)  is 
eigenvalue  of  Q  and  U  is  the  eigenvector  of  Q  corresponding  to  A j.  The  transformed 
safety  and  unsafe  specihcations  of  the  output  abstraction  can  be  obtained  using  the 
following  lemma. 

Lemma  2  Given  S(Mn)  described  by  (f),  then  S(AIk)  and  U(Mk)  defined  as  follows 
guarantee  the  safety  relation  (1): 


s(Mk)  =  {Vr  e  Rp|  {yr  ~  o)T Q{yr  —  a)  <  (R  -  AR)2}, 
U{Ml )  =  {yr  £  Rp|  (yr  -  a)TQ(yr  -a)>(R+  Afl)2}, 


The  proof  of  this  result  is  given  in  Appendix  7.2. 

We  can  see  that  the  transformed  safety  specification  of  the  output  abstraction 
is  also  an  ellipsoid  (with  smaller  radius  R  —  AR)  located  inside  the  original  ellipse 
defining  the  safety  specification  of  the  full-order  system.  Meanwhile  the  correspond¬ 
ing  transformed  unsafe  specification  is  defined  by  the  region  outside  the  larger  ellipse 
with  the  radius  R  +  AR. 

Lemma  3  Given  an  asymptotically  stable  LTI  system  Mn,  whenever  its  output  ab¬ 
straction  Mk  is  safe  or  unsafe,  it  is  sound  to  claim  that  the  system  Mn  is  safe  or 
unsafe  respectively. 

Proof  According  to  Section  3,  for  a  stable  LTI  system  Mn,  there  exists  an  output 
abstraction  Mk.  We  can  see  that,  for  any  control  input  u  £  U  and  initial  state 
x(0)  £  Xo(Mn),  there  is  a  set  of  p  output  trajectories  1  <  *  <  p  of  Mn.  From 
the  definition  of  the  output  abstraction,  there  exists  a  corresponding  initial  state 
xr(0)  £  X0(Mk )  such  that  the  abstraction  produces  a  set  of  p  output  trajectories 
y*,  1  <  i  <  p  in  which  the  distance  between  two  pair  trajectories  \\yl  —  ylr  ||  is  always 
bounded  by  a  sound  5i  all  the  time.  Moreover  from  Lemmas  1  and  2,  because  the 
transformed  specihcations  ( S(Mk )  and  U(Mk))  satisfy  the  safety  relation  (1),  thus 
when  the  set  of  p  output  trajectories  y'r,  1  <  *  <  p  of  M k  satisfies  the  transformed 
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safety  specification  S(M £),  its  corresponding  set  of  p  output  trajectories  j/i,  1  <  i  <  p 
of  Mn  also  satisfies  the  original  safety  specification  S(Mn )  all  the  time.  Consequently, 
when  the  output  abstraction  is  safe,  it  is  sound  to  claim  that  the  full-order  system 
is  safe.  A  similar  proof  can  be  given  for  the  unsafe  case.  This  completes  the  proof. 


Remark  1  It  is  useful  to  mention  that  the  procedure  of  safety  verification  for  high¬ 
dimensional  linear  systems  using  order-reduction  abstraction  can  be  done  automat¬ 
ically.  The  core  issue  is  checking  whether  the  reach  set  of  the  abstraction  returned 
by  verification  tools  such  as  SpaceEx  is  contained  inside  the  transformed  safe  set. 
This  can  be  done  in  two  steps  by  taking  advantages  of  the  existing  powerful  SMT 
solvers  such  as  Z3  [De  Moura  and  Bjprner  (2008)].  First,  the  transformed  safe  set 
can  be  described  by  a  predicate  over  the  output  variables.  Then,  we  check  whether 
the  predicate  is  satisfied  by  all  vertices  of  the  returned  reach  set  from  SpaceEx  using 
a  SMT  solver. 


We  have  studied  using  output  abstraction  for  safety  verification.  Next,  we  eval¬ 
uate  and  compare  our  approach  with  others  via  a  set  of  benchmarks. 


5  Case  Studies  and  Evaluation 

To  evaluate  the  order-reduction  abstraction  method  presented  in  this  paper,  we 
have  implemented  a  software  prototype  that  automatically  creates  output  abstrac¬ 
tions  from  full-order  systems  and  applied  it  to  a  set  of  benchmarks  [Chahlaoui  and 


Benchmark 

Property 

Initial  set  of  states 

Xo  =  {zo  E  ife(i)  <  xq (i)  <  u6(i),  1  <  i  <  n} 

Input  constraint 

«=[«!,■••  >Um]T 

Safety  specification 

*/  =  [»,■••  :Sp]T 

Motor  control 
system  (MCS) 

n  =  8 
m-  2 

p  =  2 

lb(i)  =  ub(i)  =  0,  i  =  2, 3, 4, 6, 7, 8, 

16(2)  =  0.002,  u6(2)  =  0.0025, 

16(3)  =  0.001,  u6(3)  =  0.0015. 

111  £  [0.16,  0.3], 

112  £  [0.2,  0.4], 

unsafe  region: 

0.35  <  pi  <0.4, 

0.45  <P2<  0.6. 

Helicopter 
(HELI)  [T] 

n  =  28 
m  =  6 

p  =  2 

16(i)  =  u 6(i)  =  0.1,  i  =  l,  4, 5, 6, 7, 

16(2)  =  16(3)  =  0.098,  u5(2)  =  0.11,  u6(  3)  =  0.102, 
lb(i )  =  116(1)  =  0,  8  <  i  <  28. 

«i  £  [-1,  1], 

1  <  i  <  6. 

unsafe  region: 

-1<P1<1, 

10<p2 

Building  model 
(BM)  [4] 

n  =  48 
m  =  1 

P=1 

(6(i )  =  0.0002,  116(1)  =  0.00025,  1  <  i  <  10, 

16(25)  =  -0.0001,  ii6(25)  =  0.0001, 

16(i)  =  116(1)  =  0,  11  <  i  <  48,  i  ±  25. 

111  £  [0.8,  1]. 

unsafe  region: 

0.008  <  yi 

Partial  differ¬ 
ential  equation 
(PDE)  [4] 

n  =  84 
m  =  1 

P=1 

(6(i)  =  0,  ti6(i)  =  0,  1  <  i  <  64 

16(i)  =  0.001,  116(1)  =  0.0015,  64  <  i  <  80, 

16(i)  =  -0.002,  116(1)  =  -0.0015,  81  <  i  <  84. 

111  E  [0.5,  1], 

safe  region: 

Pi  ^  12 

International 
space  station 
(ISS)  [4] 

n  =  270 
m  =  3 
p  =  3 

i6(i)  =  -0.0001,  116(1)  =  0.0001,  1  <  t  <  270. 

111  E  [0,  0.1], 

112  £  [0.8,  1], 

113  £  [0.9,  lj. 

Safe  region: 

-461pi+887p2+0.67<0, 

— 440j/i  —  898j/2  —  0.68  <  0, 
-76.7pi+997p2-0.54<0, 
898pi  -  440p2  -  0.89  <  0, 
945pi  +  326pi  -  0.95  <  0, 
-0.0005  <  p3  <  0.0005. 

FOM  [4] 

n  =  1006 
m  =  1 

P=1 

16(0  =  -0.0001,  «6(0  =  0.0001,  1  <  i  <  400 

16(i)  =  °-00()2.  UH‘)  =  0.00025,  401  <  i  <  800, 

16(i)  =  w6(0  =  0,  801  <  i  <  1006. 

HI  £  [-1,  1]. 

safe  region: 
y  1  <45 

Table  2:  Benchmarks  for  the  order-reduction  abstraction  method  in  which  n  is  di¬ 
mension  of  the  system;  m  and  p  are  the  number  of  inputs  and  outputs  respectively. 
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k 

Bisim 

Sim 

Mixed  bound 

Theoretical  bound 

m 

j 

i(s) 

N 

ei 

62 

5 

f(«) 

ei 

62 

f(s) 

MCS 

5 

2.1 

0.93 

0.0021 

0.047 

0.17 

22 *  +  2 

/0.00049) 

0.00062] 

/0.002) 

0.047) 

7000257 

0.047 

0.92 

0.0098) 

0.0093] 

(0.53 

0.53 

70547 

0.54) 

0.15 

4 

1.5 

0.64 

7UM) 

0.047, 

0.19 

22  +  2 

/0.00086) 

0.00062) 

/0.035) 

0.047 

0.036 

0.047, 

0.9 

700097 

,0.009, 

0.91 

0.91 

/0.92) 

0.92 

0.15 

HELI 

20 

0.84 

17 

.le— 0 
.3e  —  0 

y 

0.6 

24  +  6 

700072) 

0.018 ) 

5.7e  —  05 
3.2e  —  05 1 

700077 

0.018 

35 

7028) 

0.95] 

1.001 

3.001 

y 

/0.28 

0.95 

0.49 

16 

28 

12 

OMOTF 

0.0013 

0.56 

24  +  6 

/  0.0072) 
,0.018) 

0.0007  ) 
,0.00087) 

/0.0079 

,0.019 

23 

70.28) 

,0.95, 

0.029 

0.029 

r 

0.3 

0.97 

0.45 

10 

160 

8.1 

/0.024) 

0.0381 

0.55 

24  +  6 

/0.0085) 

,  0.021  ] 

/0.021) 
0.031 ) 

0.03 

0.053, 

13 

/0.27) 

0.93) 

77 

1 

7I7( 

1.9 

0.45 

BM 

25 

0.0096 

180 

0.0051 

22 

2U  +  1 

0.013 

6.2e  -  05 

0.013 

130 

0.083 

0.0072 

0.09 

1 

15 

0.069 

120 

0.005 

18 

FT 

0.012 

0.00044 

0.013 

58 

0.078 

0.084 

0.16 

0.97 

6 

0.1 

44 

0.0058 

14 

2U  +  1 

0.011 

0.00025 

0.012 

24 

0.073 

0.21 

0.28 

0.98 

PDE 

30 

0.75 

230 

N/A 

OOT 

22U  + 1 

0.033 

5.6e  - 14 

0.033 

1500 

1 

5e  — 12 

1 

1.7 

20 

0.038 

160 

N/A 

OOT 

22ll  +  l 

0.033 

3.5e  - 14 

0.033 

890 

1 

5.4e  - 12 

1 

1.7 

10 

0.086 

55 

N/A 

OOT 

FT 

0.033 

9.8e  - 13 

0.033 

520 

0.92 

2.7e  — 11 

0.92 

1.7 

6 

0.1 

42 

N/A 

OOT 

2zU  + 1 

0.033 

3.5e  -  07 

0.033 

370 

0.89 

5.5e  -  06 

0.89 

1.7 

ISS 

25 

N/A 

OOT 

N/A 

OOT 

22™  +  3 

N/A 

2.1e  —  053 

0.001 
4.6e  -  05 

N/A 

OOT 

I 

0.00043) 

0.00026 

0.00026] 

0.47 

0.47 

/0.47 

0.47 

0.47 

11 

10 

N/A 

OOT 

N/A 

OOT 

2m  +  3 

N/A 

2.4e  -  051 

5.6e  -  05 
9e  —  05 

N/A 

OOT 

000042) 

0.00022 

0.00021] 

7177) 

1.7 

\n) 

717) 

1.7 

N 

12 

FOM 

20 

N/A 

OOT 

N/A 

OOT 

2™  + 1 

N/A 

2.7e  —  07 

N/A 

OOT 

1.3 

l.le  -  05 

1.3 

48 

15 

1 LfT 

OOT 

N/A 

OOT 

2800  + 1 

N/A 

0.00021 

N/A 

OOT 

1.3 

0.0065 

1.3 

48 

10 

Tp“ 

OOT 

N/A 

OOT 

25llu  + 1 

N/A 

0.1 

N/A 

OOT 

1.3 

2.2 

3.5 

48 

Table  3:  Experimental  results  obtained  from  different  methods  in  which:  k  is  the 
dimension  of  the  output  abstraction,  5  is  the  precision,  ei  is  the  zero  input  response 
error,  ei  is  the  zero  state  response  error,  t  is  the  error  computing  time  (in  second)  and 
N  is  the  number  of  simulations.  Bisim  column  contains  the  results  of  approximate 
bisimulation  approach  proposed  by  [Girard  and  Pappas  (2007)].  Sim  column  are  the 
results  of  the  simulation-based  approach  proposed  by  [Han  and  Krogh  (2004)]. 


Van  Dooren  (2005);  Frehse  et  al.  (2011)]  presented  Table  2.  The  method  is  integrated 
in  HyST  by  calling  Matlab  related  functions.1 

Precision  and  computation  time  evaluation.  The  experiments  were  performed  using 
Matlab  2014a  and  SpaceEx  [Frehse  et  al.  (2011)]  on  a  personal  computer  with  the 
following  configuration:  Intel  (R)  Core(TM)  i7-2677M  CPU  at  1.80GHz,  4GB  RAM, 
and  64-bit  Window  7.  We  set  the  upper  limit  for  Matlab  simulation  and  SpaceEx 
running  time  at  two  hours.  In  all  experimental  result  tables,  the  term  of  “N/A” 
stands  for  not  applicable  and  “OOT”  stands  for  timeout  when  the  result  could  not 
be  computed  within  two  hours. 

Table  3  presents  the  precision  5  and  computation  times  of  the  different  meth¬ 
ods.  In  the  table,  the  “mixed  bound”  column  contains  the  bound  of  e±  computed 
by  Theorem  2  and  the  bound  of  e2  determined  by  simulation  and  the  correspond¬ 
ing  precision  5.  The  “theoretical  bound”  column  contains  the  bounds  of  e\  and  e2 
which  are  computed  using  Theorem  1  and  Theorem  3  respectively.  The  table  shows 

1  The  prototype  implementation  and  SpaceEx  model  files  for  the  examples  evaluated,  both 

before  and  after  order  reduction,  are  available  at:  http://verivital.com/hyst/pass-order- 

reduction/. 
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Benchmark 

Full  Order  System 

Output  Abstraction 

Time(s) 

Memory  (Kb) 

k 

Tl(s) 

ms) 

Total  time(s) 

Memory(Kb) 

Motor  control 
system 

27 

3048 

5 

26 

0.92 

26.9 

3044 

4 

16.7 

0.9 

17.6 

3044 

Helicopter 

287 

3052 

20 

206 

35 

241 

3052 

16 

128 

23 

151 

3048 

10 

68 

13 

81 

3048 

Building  model 

893 

3056 

25 

237.2 

130 

367.2 

3048 

15 

82.3 

58 

140.3 

3044 

6 

19.5 

24 

43.5 

3040 

Partial  differ¬ 
ential  equation 

OOT 

N/A 

30 

725.6 

1500 

2225.6 

3048 

20 

310 

890 

1200 

3048 

10 

75.2 

520 

595.2 

3040 

6 

31.9 

370 

401.9 

3040 

International 
space  station 

OOT 

N/A 

25 

254.3 

11 

265.3 

3064 

10 

72.8 

12 

84.8 

3052 

FOM  model 

OOT 

N/A 

20 

95.4 

48 

143.4 

3048 

15 

56.2 

48 

104.2 

3044 

10 

34.8 

48 

82.8 

3040 

Table  4:  Computation  cost  for  verification  process  of  the  full  order  original  LTI 
system  and  its  output  abstractions  using  SpaceEx  in  which  Xf  is  the  time  for  SpaceEx 
to  compute  the  reach  set  of  the  output  abstraction;  T2  is  the  time  for  obtaining  the 
output  abstraction;  “Total  Time”  column  states  for  the  total  time  of  verification 
process  for  the  output  abstraction,  “Memory”  column  presents  the  memory  used  for 
computing  reach  set  which  is  measure  in  kilobyte;  time  is  measured  in  second. 


that,  when  the  initial  set  of  states  X0  is  close  to  the  origin,  e.g.  BM  benchmark, 
the  bounds  of  ei  computed  by  Theorem  1  are  fairly  good  and  acceptable.  However, 
conservative  results  are  derived  when  the  initial  set  of  states  X0  is  far  from  the  ori¬ 
gin,  e.g.  helicopter  benchmark.  Theorem  3  indicates  that  the  theoretical  bound  of 
e2  depends  on  Hankel  singular  values.  This  can  be  seen  from  the  PDE  benchmark 
where  the  theoretical  bounds  of  e2  for  all  cases  of  the  output  abstraction’s  dimension 
k  are  very  small  due  to  the  fact  that  the  Hankel  singular  values  er*,  (which  are  not 
presented  here)  of  the  corresponding  balanced  system  are  very  small  (almost  equal 
to  zero)  as  k  >  5.  In  contrast,  in  the  helicopter  benchmark,  the  theoretical  bound 
of  e2  becomes  larger  when  the  lower  dimension  output  abstraction  is  obtained.  It  is 
small  when  k  =  20  because  <jj  is  small.  The  theoretical  bound  of  e2  becomes 
conservative  as  k  =  10  since  y  aj  is  large.  From  the  results  in  the  table,  we  see 
that  for  low  and  medium- dimensional  systems  (n  <  100),  the  simulation-based  ap¬ 
proach  [Han  and  Krogh  (2004)]  gives  very  tight  bounds  for  the  errors,  e.g,  the  motor 
control  system  and  helicopter  benchmarks.  This  approach  is  powerful  when  dealing 
with  systems  having  a  small  number  of  vertices  in  the  initial  set.  When  the  number  of 
vertices  increases  (e.g,  PDE  benchmark),  it  is  unable  to  apply  the  simulation-based 
approach  due  to  the  number  of  simulations  growing  exponentially.  The  approximate 
bisimulation  approach  [Girard  and  Pappas  (2007)]  integrated  in  the  Matisse  toolbox 
gives  a  good  precision  for  the  PDE  benchmark  but  very  conservative  results  for  the 
MCS  and  Helicopter  benchmarks  due  to  the  appearance  of  ill-conditioned  matrices 
in  solving  LMI  and  optimization  problems.  Combining  Theorem  2  and  simulation 
bound  of  e2  (i.e.  mixed  bound)  is  more  efficient  as  it  produces  very  tight  precisions 
for  all  benchmarks.  In  the  case  of  high-dimensional  systems  (n  >  100)  ,  the  approx¬ 
imate  bisinrulation  and  simulation-based  approaches  give  no  result  due  to  running 
out  of  time  while  our  method  can  still  be  applied.  We  can  see  that  Theorem  1  and 
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(a)  Safety  specification  S270  of  the  full  or¬ 
der  ISS  system  which  is  the  region  inside  the 
middle  green  polytopes  and  the  transformed 
safety  specifications  of  its  10-order  output  ab¬ 
straction  in  which  the  safe  region  Sf0  is  inside 
the  smallest  blue  polytopes  and  the  unsafe  re¬ 
gion  U20  is  outside  the  largest  red  polytope. 


(b)  Reachable  set  of  ( yri ,  y,2 )  of  the 
ISS’s  10-order  abstraction  M (f}  in  the 
period  of  time  [0,  20s]  and  its  safe 
region  S22  (inside  the  smallest  blue 
polygon)  and  unsafe  region  U(2  (out¬ 
side  the  red  polygon). 


Fig.  1:  The  transformed  safety  specifications  of  the  10-order  output  abstraction  of 
ISS  system  and  the  its  projected  reachable  set  on  the  (yri ,  yr2 )  plane. 


Theorem  3  have  small  computation  times  while  Theorem  2  and  the  approximate 
bisimulation  approach  require  much  more  time  to  compute  the  precision. 

Table  4  shows  the  computation  cost  of  the  verification  process  for  the  full-order 
benchmarks  and  their  different  output  abstractions.  The  bounded  times  for  running 
all  SpaceEx  models  are  set  at  £/  =  20s.  As  shown  in  the  table,  although  using  output 
abstraction  does  not  help  much  to  reduce  the  memory  used  in  verification,  it  can 
help  to  reduce  significantly  the  computation  time.  Moreover,  output  abstraction  can 
be  applied  to  check  the  safety  of  high-dimensional  systems  (e.g.  PDE,  ISS  and  FOM) 
that  cannot  be  verified  directly  using  existing  verification  tools.  Next,  we  consider 
the  whole  process  of  using  output  abstraction  to  verify  the  safety  of  the  international 
space  station  system. 


International  Space  Station  (ISS).  Verification  for  the  full-order  system  with  270 
state  variables  (denoted  by  M2 70)  may  be  difficult  for  existing  verification  tools.  Our 
approach  can  help  to  verify  safety  of  such  higli-diinensional  system  with  a  small 
computation  cost.  There  are  different  output  abstractions  that  can  be  used  to  verify 
whether  the  full-order  system  satisfies  its  safety  requirements.  I11  this  paper,  a  10- 
order  output  abstraction  is  chosen  to  check  the  safety  of  the  full-order  system.  The 
precision  S  =  10“3  x  [0.44,  0.28,  0.3]T  between  the  full-order  system  and  its  10-order 
output  abstraction  (denoted  by  M20)  is  obtained  from  the  theoretical  bound  of  ei 
and  the  simulation  bound  of  e2.  The  safety  specification  of  the  full-order  ISS  system 
£>270  is  visualized  by  the  region  inside  the  middle  blue  polytope  in  Figure  la.  The 
transformed  safety  specifications  (safe  and  unsafe  specifications)  of  the  correspond¬ 
ing  10-order  output  abstraction  respectively  are  the  region  inside  the  smallest  blue 
polytope  and  the  region  outside  the  red  polytope.  Figures  lb,  2a  and  2b  present  the 
safety  specification  transformation  and  output  reach  set  in  the  period  of  time  [0,  20s] 
computed  by  SpaceEx  for  the  10-order  output  abstraction  on  2-dinrension  axes.  I11 
the  figures,  the  regions  inside  the  middle  blue  polygons  are  the  2-dimensional  pro- 
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(a)  Reachable  set  of  ( yr2 ,  yr3 )  of  the 
ISS’s  10-order  abstraction  Mj0  in  the 
period  of  time  [0,  20s]  and  its  safe 
region  (inside  the  smallest  blue 
polygon)  and  unsafe  region  t/|3  (out¬ 
side  the  red  polygon). 


-0.5 


t  -2-1012 

\  x  10"3 

(b)  Reachable  set  of  ( yri ,  yr3 )  of  the 
ISS’s  10-order  abstraction  M in  the 
period  of  time  [0,  20s]  and  its  safe 
region  Sf3  (inside  the  smallest  blue 
polygon)  and  unsafe  region  (out¬ 
side  the  red  polygon). 


Fig.  2:  The  projected  transformed  safety  specifications  and  reachable  set  of  the  10- 
order  output  abstraction  of  ISS  system  on  the  ( yr2,yr3 )  and  ( yri,yr3 )  planes. 


jections  of  the  safety  regions  of  the  full-order  system.  The  corresponding  projected 
transformed  safety  and  unsafe  specifications  Sf0,  Uf0  of  the  output  abstraction  are 
described  by  the  regions  inside  the  smallest  blue  polygons  and  the  regions  outside 
the  red  polygons  respectively.  The  reach  sets  Rfj,  i  ^  j,  1  <  i,j  <  3  for  each  pair 
output  {yri,yrj)  of  the  abstraction  M(0  are  depicted  by  the  solid  blue  regions.  As 
shown  in  Figure  la,  the  reach  set  of  the  abstraction  is  completely  inside  its  trans¬ 
formed  safe  set.  Thus,  it  can  be  concluded  that  the  full-order  system  M2 70  satisfies 
the  safety  requirement  £270  ■  Therefore,  the  full-order  system  is  safe.  The  conclusion 
about  the  safety  of  the  full-order  system  can  also  be  given  using  projection  as  fol¬ 
lows.  Due  to  the  fact  that  one  face  of  the  polytopes  defining  the  transformed  safe 
set  is  aligned  with  one  sub-coordinate  axes  (i.e.  yriOyr2 ),  so  for  all  (i,j),  we  have 
Rfj  fl  — iiSjq  =  0  =>  Mf0  t=  Sf0.  Consequently,  the  full-order  system  is  safe.  It  should 
be  noted  that  the  above  relation  is  not  true  in  general  because  we  cannot  conclude 
a  set  is  a  subset  of  another  set  by  considering  the  relations  of  their  projections  on 
all  sub-coordinate  axes.  The  conclusion  about  safety  of  the  ISS  system  is  a  special 
case  as  one  face  of  the  safe  set  is  aligned  with  one  sub-coordinate  axes. 


6  Conclusion  and  Future  Work 

We  have  proposed  an  approach  to  verify  safety  specifications  for  high-dimensional 
linear  systems  by  verifying  transformed  safety  specifications  of  a  lower-dimensional 
output  abstraction  using  existing  hybrid  system  verification  tools.  By  reducing  the 
dimensionality,  our  method  significantly  reduces  the  time  of  reachability  computa¬ 
tions  in  the  verification  process. 

There  are  two  interesting  directions  for  future  work.  First,  the  proposed  method 
which  only  deals  with  stable  linear  systems  can  be  extended  for  unstable  linear 
systems.  Second,  our  approach  can  also  be  extended  to  more  general  hybrid  systems. 
The  main  idea  is  that  the  states  in  each  location  that  are  related  to  guards/invariants 
need  to  be  declared  as  the  outputs  of  that  location.  Then,  the  output  abstraction 
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for  each  location  can  be  obtained.  A  new  hybrid  system  is  then  constructed  based 
on  these  output  abstractions.  The  guards /invariants  of  the  new  hybrid  system  are 
obtained  by  transforming  the  former  guards / invariants  of  the  original  hybrid  system 
in  the  same  manner  of  safety  specifications  transformation  proposed  in  this  paper. 
This  approach  may  benefit  from  other  notions  of  “similarity”  between  behaviors 
(executions)  of  systems  such  as  discrepancy  functions  [Duggirala  et  al.  (2013)],  or 
conformance  degree  [Abbas  et  al.  (2014)]. 
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7  Appendix 

7.1  Proof  of  Theorem  1 

Consider  the  uncontrolled  augmented  system  x  =  Ax ,  we  have,  d(xT x)/dt  =  xT(A  + 
AT)x.  It  is  easy  to  see  that  the  controllability  grammian  and  the  observability  gram- 
mian  of  the  augmented  system  are  the  same  (denoted  as  E):  AS  +  SAT  +  BBT  =  0 
and  AT E  +  BA  +  CTC  =  0.  Combining  two  equations  yields:  (A  +  AT)E  +  I! (A  + 
At )  =  —BBt  —  CTC.  This  Lyapunov  equation  implies  that  the  real  parts  of  all 
eigenvalues  of  A  +  AT  are  necessarily  non-positive.  Because  A  +  AT  is  symmetric, 
its  eigenvalues  are  real.  Thus,  these  eigenvalues  are  either  negative  or  zero.  Note 
that  A  is  asymptotically  stable,  hence  xT(t)x(t)  — >  0  when  t  — >  oc  with  all  the 
initial  state  a:(0).  Combine  all,  we  can  conclude  that  xT(t)x(t )  is  monotonically  con¬ 
verge  to  zero  for  any  initial  state  *(0).  Using  the  monotonic  convergent  property, 
the  bound  of  the  error  ei  satisfies:  ||e|(t)||  =  ||Cyi||  <  ||Ci||  ||x||  <  ||(7i||  ||x(0)||  < 
||C,||  -supx(0)eXo  ||x(0)||  . 


7.2  Proof  of  Theorem  2 

Consider  the  uncontrolled  augmented  system  (i.e.  u  =  0),  let  V{x (t))  =  x(t)T Px(t), 
we  have  V(x(t))  =  x(t)T (AT P  +  PA)x(t).  Assume  P0  is  the  solution  of  the  opti¬ 
mization  problem  in  Theorem  2.  Because  of  (AT PQ  +  PqA)  <  0,  then  V(x(t))  < 
U(x(0)  =  x(0)T Pox(0).  Note  that  ||e^(t)||“  =  xTCfCiX,  1  <  i  <  p-  Since  we  also 
have  CjCi  <  P0,  the  bound  of  the  error  satisfies  1 1  (t)  1 1  <  y/x(0)T  Pox(0). 


7.3  Proof  of  Lemma  1 

From  the  dehnition  of  output  abstraction,  we  have: 

otijVvj  ~  \otij |5j  <  ctijUj  <  aijyrj  +  \oiij\6j  =>  Pyr  +  &2  <  ry  +  P  <  Pyr  +  Pi. 

Thus,  5(M|)  and  U(M f)  defined  by  (3)  satisfy  the  safety  relation  (1),  which 
completes  the  proof. 

7.3.1  Proof  of  Lemma  2 

Let  y  =  E(y  —  a),  yr  =  E(yr  —  a).  We  have: 


(y  ~  a)TQ{y  ~a)  =  yT Ay  =  Y^l=1 


(5) 


From  the  definition  of  output  abstraction  (Dehnition  1),  it  is  easy  to  see  that: 
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Using  (6)  and  the  Cauchy-Schwarz  inequality  yields: 


J2]=1  Xi(yt  ~ y^2  -  =  X^=1  Xi5^ 


'y  '  ^  2A jyTi ( Dj  Vn)  —  2Z\j;y  ^  ^  (7) 

2Xiyi^i  ~  yi)  ^  2AR\ly^P.  ^  ^iVi- 

From  (7),  the  following  inequalities  are  true: 

X^=1  A^  -  ^Y^i=1  Xiyri  +  ^«)2’  X^=1  Ai^2»  -  (y^=1Ai^2  +  Zii?)2-  (8) 

From  (5)  and  (8),  we  have: 

V(y  -  a)TQ(y  -  a)  <  \J {yr  -  a)TQ{yr  -  a)  +  AR, 

V(y-  a)TQ(y  -  a)  >  \/(yr  -  a)TQ(yr  -  a)  -  AR. 


0) 


Using  (9),  we  can  conclude  that  and  S(Mji)  defined  in  Lemma  2  satisfy 

the  safety  relation  (1),  which  completes  the  proof. 
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Abstract —  The  reachable  set  estimation  and  control  problems 
for  continuous-time  switched  linear  systems  are  addressed  in 
this  paper.  First,  a  general  result  on  reachable  set  estimation 
for  switched  system  is  proposed  based  on  a  Lyapunov  function 
approach.  Then,  with  the  help  of  a  class  of  time-scheduled 
Lyapunov  functions,  a  numerically  tractable  sufficient  condition 
ensuring  the  system  state  bounded  in  a  prescribed  set  is  derived 
for  switched  systems  under  dwell  time  constraint.  Moreover,  a 
time-scheduled  state  feedback  controller  is  designed  to  ensure 
the  state  trajectories  of  the  closed-loop  system  are  confined  in 
a  prescribed  set.  Finally,  a  networked  control  system  subject 
to  packet  dropouts  is  modeled  as  a  switched  system  with  dwell 
time  constraints,  and  the  controller  design  problem  is  studied 
as  an  application  of  our  results. 

I.  Introduction 

Switched  systems  have  emerged  as  an  important  class 
of  hybrid  systems  and  represent  an  active  area  of  current 
research  in  the  field  of  control  systems  [1] — [3].  A  switched 
system  is  composed  of  a  family  of  continuous  or  discrete¬ 
time  subsystems  along  with  a  switching  rule  governing  the 
switching  between  the  subsystems.  Generally,  the  stability 
and  stabilization  problems  are  the  main  concerns  in  the  field 
of  switched  systems.  It  has  been  established  that  Lyapunov 
function  techniques  are  effective  to  deal  with  stability  and 
stabilization  problems  for  switched  systems,  e.g.  see  [4]— [8] . 
Combining  multiple  Lyapunov  function  (MLF),  the  dwell 
time  and  average  dwell  time  properties  of  relatively  slowly 
switched  systems  have  been  investigated  [9]— [  15] . 

Reachable  set  estimation  aims  to  derive  a  closed  bounded 
set  that  constrains  all  the  state  trajectories  generated  by  a 
dynamic  system  with  a  prescribed  class  of  initial  state  set 
and  inputs.  Reachable  set  estimation  is  not  only  of  theoret¬ 
ical  interest  in  robust  control  theory  [16],  but  also  closely 
related  to  practical  engineering  for  the  safety  verification 
problems  [17].  In  some  early  work,  reachable  set  bounding 
was  considered  in  the  context  of  state  estimation  and  it  has 
later  received  a  lot  of  attention  in  parameter  estimation,  see 
[18]  and  references  therein.  Recently,  employing  ellipsoidal 
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techniques  based  on  Lyapunov  function  approaches  to  esti¬ 
mate  the  reachable  sets  for  different  class  of  systems  attracts 
many  researchers’  attention.  In  the  framework  of  bounding 
ellipsoid,  the  quadratic  Lyapunov  function  has  played  a 
fundamental  role  in  the  reachable  set  estimation  problem, 
and  it  has  been  developed  to  time-delay  systems  [19]— [21], 
singular  systems  [22],  discrete-time  switched  systems  [23]. 
However,  according  to  the  best  of  the  authors’  knowledge, 
the  reachable  set  estimation  for  continuous-time  switched 
systems  with  constrained  switching  law,  has  not  been  fully 
investigated,  and  it  motivates  our  study  in  this  paper. 

In  this  paper,  the  problems  of  reachable  set  estimation  and 
control  synthesis  for  continuous-time  switched  linear  systems 
will  be  investigated.  First,  a  general  result  based  on  Lyapunov 
function  approach  is  presented.  Then,  under  the  framework 
of  dwell  time  and  with  the  help  of  a  class  of  time-scheduled 
quadratic  Lyapunov  functions,  a  linear  matrix  inequality 
(LMI)  based  sufficient  condition  is  proposed  to  estimate  the 
reachable  set.  For  the  control  synthesis,  a  time-scheduled 
feedback  controller  is  designed  to  ensure  the  state  trajec¬ 
tories  being  contained  in  a  prescribed  set  and,  moreover, 
an  optimization  problem  is  formulated  to  obtain  an  optimal 
controller  gain  to  make  the  reachable  set  of  closed-loop 
system  as  small  as  possible.  As  an  application  of  our  result, 
the  control  problem  for  a  networked  control  system  with 
package  dropouts  is  studied.  Based  on  our  derived  approach, 
the  controller  can  be  designed  with  an  attempt  to  constrain 
state  trajectories  in  a  prescribed  bounding  ellipsoidal  region. 

Notation:  The  notations  in  this  paper  are  fairly  standard. 
S”xn  is  the  set  of  real  symmetric  positive  definite  n  x  n 
matrices.  In  symmetric  block  matrices,  we  use  *  as  an  ellipsis 
for  the  terms  that  are  introduced  by  symmetry.  diag{-  •  •  } 
denotes  a  block-diagonal  matrix  and  int[-]  rounds  the  element 
to  the  nearest  integer  towards  zero. 

II.  Preliminaries  and  Problem  Formulation 

Let  us  consider  a  continuous-time  switched  linear  system 
in  the  form  of 

x{t)  =  Aa{t)x(t)  +  BUtCr{t)uj{t)  +  Bu^{t)u(t)  (1) 

where  x(t)  £  are  the  state  of  the  system,  and  the  initial 
state  xq  is  assumed  to  be  settled  in  a  bounded  ellipsoid  as 

x0  £  Xo  =  {an  €  R"x  |  xjR0xo  <l,Ro£  (2) 

and  u)(i)  £  KnuJ  is  the  disturbance  input  vector  which  is 
assumed  to  satisfy  the  following  ellipsoidal  constraint 

u(t)  £  W  =  {to  £  IT"  |  utRuoj  <  1  ,Ru£  (3) 
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and  u(t)  G  R"“  is  the  control  input  to  be  designed. 

Define  an  index  set  Ml  =  {1,  2, . . . ,  TV},  where  N  is  the 
number  of  modes  and,  a  :  R>o  — »  Ml  denotes  the  switching 
function,  which  is  assumed  to  be  a  piecewise  constant 
function  continuous  from  right.  The  switching  instants  are 
expressed  by  a  sequence  S  =  {4}fc£N>  where  to  denotes  the 
initial  time  and  tk  denotes  the  fcth  switching  instant.  Then, 
we  define  2}  =  {t  G  R>o  |  cr(f)  =  i,i  G  Ml}  to  denote  the 
activation  time  interval  for  ith  mode. 

The  first  problem  considered  in  this  paper  is  the  reachable 
set  estimation  problem  for  switched  system  (1)  with  control 
input  u(f)  =  0,  and  the  initial  state  satisfying  (2),  disturbance 
input  satisfying  (3).  The  reachable  set  is  defined  as 

TZx  =  {x£Rnx  |  x(t),x0,u(t)  satisfy  (1),  (2),  (3)}  (4) 

Then,  the  mode-dependent  state  feedback  controller  is 
considered,  which  has  a  time-scheduled  structure  as 

u(t)  =  KaW(t)x(t)  (5) 

Substituting  above  controller  (5)  into  system  (1),  the 
closed-loop  system  becomes 

x(t)  =  Aa{t)(t)x{t)  +  BUi<rit)u(t)  (6) 

where  Aa(t){t)  =  Aa(t)  +  Bu^{t)K(j(t)(t). 

The  control  objective  is  to  ensure  the  state  trajectory  x(t) 
contained  in  a  given  set 

Hx  =  {x  G  ln”  |  xtRxx  <  1  ,RX£  (7) 

The  above  two  problems  are  the  main  concerns  in  this 
paper.  In  the  rest  of  this  paper,  the  reachable  set  estimation 
problem  will  be  studied  at  first,  then  based  on  the  reachable 
set  estimation  results,  the  state  feedback  controller  design 
problem  will  be  addressed. 

III.  Reachable  Set  Estimation 
A.  General  Lemma 

First,  a  general  lemma  is  presented  to  introduce  the  main 
idea  to  determine  the  over  approximate  set  1ZX  for  switched 
system  (6),  note  that  switched  system  (1)  with  u(t)  =  0  is  a 
particular  case  of  Aj(f)  being  time-invariant. 

Lemma  1:  Consider  switched  system  (6)  under  initial  state 
condition  (2)  and  disturbance  input  condition  (3).  If  there 
exist  a  family  of  Lyapunov  functions  Vj  :  R"x  — >  R>o, 
i  G  Ml ,  satisfying  Vj(0)  =  0  and  Vi(x)  >  0,  Vx  ^  0, 
V*  G  Ml,  and  scalars  a  >  0,  0  <  /3  <  1  such  that 

FAt)  <  0,  Vf  GXi,Vi  G  Ml  (8) 

Gij{tk )  <  0,  Vffc  G  S,i  ±  j,Vi,j  G  Ml  (9) 
Vi(x0)  <  Xq  R0x0,  Vi  G  Ml  (10) 

where  Fj(i)  =  Vi(x(t))  +  aVi(x(t))  —  au;T(f)i?wa;(f)  and 
Gij(tk)  =  Vi(x(t£))  -  pVj(x(tf))  +  p  -  1.  Then,  the 
reachable  set  7 Zx  satisfies  1ZX  C  1ZX  =  {xG  R"x  |  Vfx)  < 
1  ,i  G  Ml}. 

Proof:  Define  the  following  Lyapunov  function  as 


where  :  R>0  ->  {0,1}  and  =  1  is  the 

indicator  function  indicating  the  active  modes  at  t. 

First  we  consider  any  t  G  [tk,tk+i)  C  li,  Vi  G  Ml.  (8) 
implies  V(t)  <  —aV(t)  +  awT(f)i?ww(f),  t  G  [4,4+ 1). 
Multiply  both  sides  of  this  inequality  with  ea^~tk^  and  then 
integrating  it  over  [44)-  we  hare  V(t)  <  e~a^~tk^V (t^)  + 
J*k  e_a(t_s)a;T(s)i?CJu;(s)ds.  Due  to  wT(f)i?wu;(f)  <  1, 
Vf  G  R>o,  we  have  the  following  result 

V{t)  <  e"a(‘-tfc  V(f+)  +  J  e-a(t-s)ds 

=  e~a{t-tk)V(t+)  +  1  -  e-a(*-tk) 

and  it  can  be  rewritten  to 

V(t)  -  1  <  e-a^-tk\V(t+)  -  1),  t  G  [4,4+t)  (13) 

Next,  we  consider  tk  G  S.  From  (9),  we  can  obtain 
V(fjJ')  <  pV(tf)  +  1  —  p,  tk  G  S,  which  equals  to 

V(t+)  -  1  <  p(V(tf)  -  1),  4  6  5  (14) 

Combining  (13)  and  (14),  for  Vf  G  R>o,  it  can  be  obtained 

V{t)  -  1  <  •  •  •  <  (0Num(t-to)g-a(t-to')(y(fo)  _  i),  where 

Num(f  —  4)  is  the  number  of  switchings  in  [fo,f).  Due  to 
a  >  0  and  0  <  P  <  1,  it  means  that  V(t)  —  1  <  V(fo)  — 
1,  Vf  G  R>o-  Moreover,  (10)  implies  V(f0)  <  Xq  RoXo  <  1, 
and  it  yields  V[t )  <  1,  Vf  G  R>o  holds,  so  x(t)  G  1ZX,  Vf  G 
R>o,  where  Rx  =  {iG  R"x  |  x)  <  1, i  G  Ml}.  ■ 

Although  Lemma  1  provides  a  general  framework  to  deal 
with  the  reachable  set  estimation  problem,  it  is  trivial  in 
actual  use,  since  it  does  not  provide  any  available  computa¬ 
tional  techniques  for  the  construction  of  Lyapunov  functions 
Vi(x(t)),  i  G  Ml  and  moreover,  the  proposed  condition  (9) 
requires  us  to  check  the  values  of  Lyapunov  functions  at 
every  the  switching  instant  4  6  S.  However,  the  switching 
instant  sequence  S  usually  cannot  be  specified  in  advance, 
and  it  is  impossible  to  check  Lemma  1  for  all  switching 
instants  tk  in  the  case  of  k  — >  oo. 

B.  Time-Scheduled  Multiple  Lyapunov  Functions 

Based  on  Lemma  1,  we  particularly  consider  a  class  of 
switched  system  with  dwell-time  constraint. 

Definition  1:  Given  a  switching  signal  function  a(t)  with 
a  generated  switching  sequence  S ,  Tm;n  =  inffcgN{4+i  —  4} 
is  called  the  minimum  dwell  time  of  <r(f).  DTmin  =  {a  \  a  : 
R>o  — >  Ml,tk+i  —  tk  >  Tmin,Vfc  G  N}  denotes  the  set  of 
all  switching  policies  with  dwell  time  greater  than  rm;n. 

Then,  inspired  by  [11],  [12],  [15],  we  consider  a  class  of 
time-scheduled  multiple  Lyapunov  functions  as  follow 

Vi(x(t))  =  xT (t)Pi(t)x(t),  t  G  R>o,  i  G  Ml  (15) 

where  P,  (t)  G  §™xXrixj  i  £  Ml  have  the  following  structure: 

Consider  the  interval  [4,4  +  rmin),  we  divide  it  into  L 
segments  described  as  Ck,q  =  [tk  +  4+  ffc  +  0q+i),  1  = 
0, 1, . . . ,  L  —  1  of  equal  lengths  h  =  Tmm/L,  and  then  do  =  0 
and  0q  =  qh  =  qrmin/L.  We  consider  a  class  of  continuous 
matrix  function  P,(f),  f  G  [ffc,  ffc  +Tmin)  chosen  to  be  linear 
41)  within  each  segments  £fc,9,  q  =  0, 1, . . . ,  L  —  1.  Explicitly, 
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we  can  see  that  U™=o  £fc,n  =  [4,4  +  Tmin)  and  £fc)„  n 
4fc,m  =  0,  n  /  m.  Letting  Pij9  =  Pj(4  +  9q),  then  since 
the  matrix  function  P/t)  is  piecewise  linear  in  [4, 4+Tmin), 
it  can  be  expressed  in  terms  of  the  values  at  dividing  points 
using  a  linear  interpolation  formula,  that  is,  for  0  <  n  <  1, 
q  =  0,1, . . . ,  L  —  1, 

Pi(t )  =  P/p)  =  (1  -  H)Pi,q  +  PPi,q+ 1,  t  G  Ck,q,  1  G  Ad 

(16) 

where  /.t  =  L(t  -  tk  -  6*9)/rmin. 

As  a  result,  the  continuous  matrix  function  P/t)  G 
S"xX"x,  7  G  Ad  can  be  completely  determined  by  Piq  G 
S"xX"x,  q  =  0,l,...,L,  i  G  Ad,  in  interval  [4, tk  +  rmin). 

Then,  due  to  [4, 4  +  rmin)  Q  [4, 4+i),  for  the  remaining 
time  in  [4,4+i)  denoted  by  £fc>i  =  [4,min,4+i). 
i  G  Ad  is  set  to  be 


P,:(f)  =  Pj,L,  t  G  i  G  Ad 


(17) 


In  summary,  the  Pj(i),  i  G  Ad  in  Lyapunov  function  in 
(15)  is  defined  as 

P.(t)  =  {P^),  jG£fc>„  9  =  0,1,  1  (18) 

(  Pi,L ,  t  G  £fc,L 

where  //  is  defined  in  (16). 


C.  Reachable  Set  Estimation  under  Dwell  Time  Constraint 


Now,  we  are  ready  to  propose  out  main  result  as  follows. 

Theorem  1:  Given  dwell  time  rm in  >  0  and  consider 
switched  system  (1)  with  <j(f)  G  f>Xmin  under  initial  state 
condition  (2),  disturbance  input  condition  (3)  and  u(t)  =  0.  If 
there  exist  a  set  of  matrices  Pi  q  G  §™xXna:,  q  —  0, 1, . . . ,  L, 
i  G  Ad  and  a  scalar  a  >  0  such  that  for  Vi,  j  G  Ad 


+  'Fj  q 

P 

i,q 

* 

qRu 

A0, 

<7  =  0,.. 

■,L-1 

(19) 

1  +  ^  i,q 
Bt  P 

u ,i±  i,q 

* 

qRoj 

A0, 

<7  =  0,.. 

.,£-1 

(20) 

BZ,iPi,L 

* 

cxR^j 

A  0 

(21) 

-Pj.o  —  -Pi,i  A  0,  7  j  (22) 

P;.o  —  Pq  "5  0  (23) 


where  =  Aj  Pi:q  +  P^qAt  +  aPi:Q  and  d/i)3  = 

L(Pi  q+ 1  —  Pi,q)/rm in-  Then,  the  reachable  set  Px  C  Px  = 
{a;  G  K"x  |  Pi,qx  <  1,  q  =  0, 1, . . . ,  L,  i  G  Ad}. 

Proof:  Construct  Lyapunov  function  as 


C(<)  =  .cMb(t)xT{t)Pi(t)x(t)  (24) 

z — 'iEM 

where  P/t),  7  G  Ad,  is  defined  by  (18)  and  £*(•)  is  defined 
same  as  (11). 

First,  let  us  consider  F/t)  =  V(t)  +  aV(t)  — 
auT  (t)Ruw(t),  which  can  be  rewritten  to 


Fi(t)  =  xT(t) 


Ei(t)+Pi(t)  * 
BZ,iPi(t)  ~aR 


where  xT(f)  =  [xT(f)  wT(f)]  and  S*(i) 
P%(t)Ai  +  aPft). 


X(t)  (25) 
=  Aj  Pi(t )  + 


TABLE  I 

Computational  Complexities  of  Theorem  1  with  a  Fixed  a 


Number  of  Decision  Variables 

LMI  Constraints  Size 

nN(L  +  1  )(n  +  l)/2 

2nN(N  +  2L  +  1) 

Suppose  a(t)  =  i,  t  G  Ck,q,  q  =  0, . . . ,  L  —  1,  one  has 


Ei(t)  +  Pi(t)  * 

BZ,iPi{  *)  ~aR 


(1  -  p)Uitl  +  (26) 


where  Id,  !  = 

^i,q+ 1  +  '*i,q+ 1 
BZ,iPi,  9+1 


“i  ,qr  +  * 

BZ,iP‘,q  —aRL 

* 

cxRtjj 


and  ni)2  = 


Furthermore,  we  can  see  pi{t)  =  (Pj>g+i  —  Piq)p, 
t  G  Pfc,g,  <7  =  0, . . . ,  L  —  1,  and  because  of  fi  =  L{t.  — 
tk  -  0q)/r min,  it  implies  /i  =  P/rmi n,  leading  to  P,(f)  = 
(E'i.g,  t  G  £fc,g,  g  =  0, . . . ,  L  -  1,.  By  (19),  (20),  it  leads  to 


Fi(t)  <0,  Vt  G  M  £fe,ra  =  [4,4  +  Tmin) 

^n— 0 


(27) 


Then,  we  consider  t  G  Ck,L ■  Since  P,(t)  =  Pel,  t  G  Ck,L, 
we  have  P,(t)  =  0,  Vt  G  £a,,l,  thus  (21)  guarantees  that 
Pj(t)  <0,  Vt  G  £fc,L.  Together  with  (27),  we  can  conclude 
that  Pj(t)  <  0,  Vt  G  X Vi  G  Ad,  which  means  (8)  in 
Lemma  1  holds. 

Next,  (22)  ensures  (9)  holds  with  /)  =  1  and  (23) 
guarantees  (10)  holds.  Therefore,  we  have  the  reachable  set 
Px  C  Px  =  (a;  G  I"1  |  xT Pitqx  <  1,  q  =  0, 1, , . . ,  L,  i  G 
Ad}  by  Lemma  1.  ■ 

Remark  1:  Parameter  L  implies  the  number  of  segments 
consisting  of  the  dwell  time  interval  [4, 4  +  Tmin).  A  larger 
L  yields  a  finer  division  of  [4,4  +  rmin),  and  a  less 
conservative  result  can  be  consequently  obtained,  which  will 
be  demonstrated  by  a  numerical  example  later.  However, 
the  computational  cost  increases  as  L  grows,  since  a  larger 
L  inevitably  introduces  more  decision  variables  and  LMI 
constraints,  see  Table  I  for  the  computational  complexity 
analysis  for  Theorem  1. 

The  set  Px  is  usually  expected  to  be  as  small  as  possible 
to  achieve  a  precise  estimation  of  reachable  set  P„.  Based 
on  Theorem  2,  one  may  add  an  additional  constraint  that 


P,.q  Pel,  e  >  0,  Mq  =  0, 1, . . . ,  L,  VI  G  Ad  (28) 

which  implies  that  exT  (t)x{t)  <  xA  (t)Pitqx(t)  <  1,  namely 
x(t)  G  B{ 0, 1/Ve)  =  {x  G  R"  |  ||.c||  <  l/sfe},  Vt  G  K>0, 
so  we  have  to  maximize  e  to  obtain  a  smallest  reachable  set 
with  respect  to  e.  Given  an  L,  the  smallest  ball  £>(0, 1/s/e)  = 
{x  G  K"  |  ||x||  <  1/Ve}  containing  the  trajectories 
of  state  x(t)  in  the  framework  of  our  approach  can  be 
obtained.  Based  on  Theorem  1,  an  optimization  problem  can 
be  formulated  by  adding  (28)  with  (19)— (23)  as  follows 

max  e  s.t.  (28)  and  (19)  —  (23)  (29) 


In  the  extreme  case  with  L  =  0,  Piiq,  shrinks  to  P, , 
moreover,  due  to  (35),  we  have  to  choose  P,  =  Pj,  i  /  j. 
Thus,  Theorem  2  is  reduced  to  the  following  corollary. 
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Corollary  1:  Consider  switched  system  (1)  under  initial 
state  condition  (2),  disturbance  input  condition  (3)  and 
u(t)  =  0.  If  there  exist  a  matrix  P  £  anc|  a  scalar 

a  >  0  such  that 


AjP  +  PAi  +  aP  * 
?T  u  -aR, 


B'  P 

U),l 


~<  0,  Vi  e  M  (30) 
P  —  Ro  0  (31) 


Then,  the  reachable  set  1ZX  C  iZx  =  {x  £  |  xT  Px  < 

1}. 

Remark  2:  Corollary  1  is  actually  the  straightforward  re¬ 
sult  derived  based  on  the  well-known  common  Lyapunov 
function  approach.  It  can  be  observed  that  there  is  no 
restriction  for  the  dwell  time,  this  means  that  it  can  be  used 
for  arbitrary  switching  case  which  includes  broader  classes 
of  switching  signals,  however,  the  cost  is  the  increase  of 
conservativeness  of  the  estimation  results. 


IV.  Time-Scheduled  Feedback  Controller  Design 


In  this  section,  the  controller  design  problem  will  be  con¬ 
sidered  in  the  framework  of  dwell  time.  Based  on  Theorem 
1,  the  following  result  can  be  derived  for  controller  design. 

Theorem  2:  Given  dwell  time  rm;n  >  0  and  consider 
switched  system  (1)  with  cr(t)  £  27Tmin  under  initial  state 
condition  (2)  and  disturbance  ui(t)  satisfying  (3).  If  there 
exist  a  set  of  matrices  Si>q  £  §^xn-,  Xi>q  £ 
q  =  0,1, ...  ,L,  i  £  At  and  a  scalar  a  >  0  such  that  for 
Vi,  j  £  M 


Vi/ 

,i 

* 

cT 

Y 

o' 

II 

■  ,L  —  1  (32) 

VT/ 

^  i,q 

* 

—aRu 

-<0, 

cT 

II 

.  ,L  —  1  (33) 

l— ' i,L 

BX 

* 

—aRcj 

-<  0 

(34) 

SjtL  —  Sifi  -<  0, 

*7 lj 

(35) 

Ro 1  -  Sit o  ^  0 
Si,q  -  Rf1  -<  0, 

<7  =  0,.. 

(36) 

■  ,L  —  1  (37) 

where  ^ijq  A^S^^q -(- Si  qAj  T Bi^uXj^qP X^qB^^u-\-  cxS^q 
and  \D i  q  =  L(S1,;  9+i  —  S.xq) / Tm-m.  Then,  the  closed-loop 
system  (6)  with  controller  gain  Ki(t)  =  Xj(f)S'“1(f)  has  a 
reachable  set  1ZX  C  ftx  ={ie  K"x  |  xT Rxx  <  1},  where 
Sift)  and  Xft)  are  given  by 


Si(t) 

Xi(t) 


f(l  fx)Si^q  +  /-lSi^q+l  t  £  Ck,q 
\  Si,L  t  €  Ck,L 

f  ( 1  P)Xi,q  T  qiX-l  q  t  £  Dk,q 
\  Xii  t  £  Ck,L 


(38) 

(39) 


where  =  Lft  —  ffc)/rm in  —  q  and  q  is  determined  by 


fint  [L(t-tk)/T, 
Q  1  L 


Proof: 


(40) 


0  <  m  <  L 
q  >  L 

Since  Slxl  >-  0,  it  implies  Sift)  defined  by 


(60)  is  positive  definite,  and  thus  we  have  St  i(<)  >-  0. 


Then,  a  Lyapunov  function  for  closed-loop  system  (6)  can 
be  constructed  as  follows: 


V(t)  =  &(.t)xT(t)St  1(t)x(t)  (41) 

where  £,;(•)  is  defined  same  as  (11). 

Substituting  Xft)  =  IT  ft)  Sift),  (32)-(34)  ensure  the 
following  inequality  holds 

'  Aft)Sft)  +  Sft)AJ (f)  +  aSft)  -  Sft)  *  1 

Bu,i  -aRu  \ 

(42) 

Multiplying  both  side  of  (42)  by  diag-fST1^),  1}  and 
using  5r1(f)  =  -S~1(t)Si(t)S~1(t),  we  have 


s i(t )  * 

B^Sfft)  - aR 


(43) 


where  Eft)  =  Aj (t)St  \t)  +  S.t  \t)Aft)  +  \t)  + 

aSi  1(f).  It  implies  (8)  in  Lemma  1  holds. 


Then,  we  consider  (35)  and  (36).  If  (35)  holds,  it  equals 
-<  0  by  Schur  complement.  Then, 


to  $  = 


_c-t 


-S. 


i,  0 


further  considering  the  Schur  complement  of  <1»,  we  obtain 
Sfo  ~  Sjl  -<  0  implying  (9)  in  Lemma  1  holds  with  0  =  1. 
Similarly,  (10)  can  be  guaranteed  by  (36).  Thus,  we  have 
the  reachable  set  1ZX  C  TZX  =  {x  £  M"x  |  xT  Sfqx  < 
1, q  =  0,1, ...  ,L,i  £  A 4}  by  Lemma  1.  Finally,  from  (37), 
we  have  xT Rxx  <  xTSfqx  <  1,  q  =  0, 1, . . . ,  L,  i  £  Ai, 
which  implies  7 Zx  C  TZX  =  {a:  £  Rnx  |  xT Rxx  <  1}.  ■ 

Remark  3:  In  order  to  obtain  a  optimized  controller  for 
the  smallest  reachable  set  estimation  for  closed  loop  system, 
we  can  add  the  following  constraint 


Si>q-6I~<0,  d  >  0,q  =  0, ...  ,L,i  £  M  (44) 

The  above  inequality  ensures  the  x(t)  £  B{ 0,  y/S)  =  {x  £ 
1"  |  ||x||  <  y/6}.  Given  an  L.  the  smallest  ball  B( 0,  \/S) 
containing  the  reachable  set  1ZX  can  be  obtained  by  the 
following  optimization  problem 

min  6  s.t.  (44)  and  (32)  —  (36)  (45) 

Corollary  2:  Consider  switched  system  (1)  under  initial 
state  condition  (2)  and  disturbance  u(t)  satisfying  (3).  If 
there  exist  matrices  S  £  §fxXnx,  Xt  £  K"x,xnx,  i  £  Ai  and 
a  scalar  a  >  0  such  that 

BX  —aRw  *  °’  e  M  (46) 

Rfl  <S<  Rf1  (47) 

where  Ei:q  =  AiS  +  SAj  +  BU:iXi  +  Xj BX  +  aS.  Then, 
the  closed-loop  system  (6)  with  controller  gain  Ki  =  X,S 
has  a  reachable  set  7ZX  C  Hx  =  {i£  |  xT Rxx  <  1}. 

Proof:  It  can  be  easily  proved  by  letting  L  =  0  in 
Theorem  2,  so  the  proof  is  omitted  here.  ■ 

Though  Corollary  2  provides  constant  feedback  gains  I\  j, 
i  £  M  which  does  not  require  online  computation  as 
Kift ),  i  £  A4  do.  This  feature  is  more  convenient  for 
controller  realization  in  practice,  but  the  conservatism  grows 
in  comparison  with  the  case  of  L  >  0. 
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u(t) 


y(t)orx(t) 


\  Tj  Network  T2  \ 


Feedback 

controller 


Fig.  1.  Packet  dropouts  in  networked  control  system 


V.  Application  in  Networked  Control  Systems 

Consider  a  networked  control  system  with  packet  dropouts 
in  both  forward  and  backward  channels,  where  the  packet 
dropouts  can  be  modeled  as  switches  open  behavior,  which  is 
illustrated  in  Fig.  1.  When  7)  is  closed,  the  controller  output 
is  successfully  transmitted  to  the  actuator;  whereas  when  it 
is  open,  the  output  of  the  switch  becomes  zero  and  a  packet 
is  lost,  and  we  have  in  this  case  u(t)  =  0.  The  situation  is 
the  same  for  the  backward  channel.  In  absence  of  packet 
dropouts,  the  state  feedback  controller  works  well  during 
interval  Fi,*,  =  [t2kl  t2fc+i)>  k  £  N.  However,  due  to  the 
occurrence  of  packet  dropouts,  the  controller  is  considered 
to  be  not  available,  namely  u(t)  =  0,  in  the  time  interval 
r 2,k  —  [t2k+lit2k+2),k  £  N. 

Assumption  1:  The  following  assumptions  are  made: 

1)  There  exists  a  uniform  lower-bound  rmjn  on  the  lengths 
of  r1)fc)  k  £  N,  that  is  f2fe+i  —  t2k  >  Tmin,  Vfc  £  N. 

2)  There  exist  a  uniform  upper-bound  tjjmax  on  the  lengths 
of  r2,fe,  k  £  N,  that  is  t2fc+2  ~t2k+i  <  ip  max?  Vfc  £  N. 

The  plant  we  consider  is  a  linear  system 

x(t)  =  Ax(t)  +  Buui(t)  +  Buu(t)  (48) 

and  the  controller  is  considered  to  be  u(t)  =  K(t)x(t),  t  £ 
k  |  In  summary,  the  networked  control  system  with  packet 
dropouts  can  be  described  as  follows 


x(t)  =  Aa{t)(t)x(t)  +  Buw(t) 


(49) 


where  A\(t)  =  A  +  BuK(t)  and  A2(t)  =  A,  and  the 
switching  function  cr(t)  is 


r(t )  = 


1  t  £ 

2  t  £  r2  h 


(50) 


Theorem  3:  Under  Assumption  1  and  consider  networked 
control  system  (48)  under  initial  state  condition  (2)  and 
disturbance  uj(t)  satisfying  (3).  If  there  exist  a  set  of  matrices 

Sitq  £  S^xn*,  X<q  £  R”“xn*,  q  =  0,1,...,  L,  i  =  1,2 

and  a  scalar  a  >  0  such  that 


A,q  ~  ^i,g  * 

Bl  -aRu 

-1,9+1  —  ^1,9  * 

Bj,  —aRn 


■=■1  ,L  * 

B Z  -uRu. 

^2,q  ~  4^2,9 

BJ, 


A  0 


* 

—0:RU 


A  0,  q  =  0,...,L-  1  (51) 

A  0,  q  =  0,...,L-  1  (52) 
(53) 

A0,  q  =  0, . . . ,  L  —  l  (54) 


-2,g+l 

BT, 


-  ^2,9  * 

cxRtjj 

Sl,L  —  S 2,0  0 

<52,9  —  •S'i.o  A  0,  g  =  0,.. 
+0  A  0,  *=1,2 


^o_1  -  £ 


Si,q  —  Rx  1  A  0,  *  =  1,2,  q  =  0,...,L 


A  0,  q  =  0,...,L-  1  (55) 
(56) 

L  (57) 

(58) 

(59) 


where  ^i,9  =  AS\tq  +  Si,qAT  +  BuX\^q  +  Xq  qB^  + 
otS\)Q,  =  ASi>q  +  S2,qAT  +  aS2^q  and  'kl  9  = 

L(Shq+1  —  Sl^/Tmin,  ^2,q  =  L{S2,q+l  ~~  S2,q )  /^max- 
Then,  the  closed-loop  system  (49)  with  controller  gain 
K(t )  =  2fi(f)S'j_1(f),  t  £  Ti^.  has  a  reachable  set  1ZX  C 
TZX  =  {x  £  |  xT Rxx  <  1},  where  Si(t )  and  X\ (t)  are 

SXt)  =  ((1 "" Sl’q  +  €  (60) 

[  i>i,L  t  £  Lk,L 

X1{t)  =  hl~^X'’q  +  ^Xl’q  (61) 

[  Ai tL  t  £  Lk,L 

where  p  =  L(t  —  i2fc)/rmin  —  q  and  q  is  determined  by 


_  /  int  [L(t  -  t2k)/Tmin\  0  <m<L 
q  \  L  q>L  K  ’ 

Proof:  By  the  similar  guidelines  in  Theorem  2,  condi¬ 
tions  (51),  (52)  and  (53)  ensures  that  (8)  in  Lemma  1  holds 
for  interval  Ti  fc,  and  (54),  (55)  guarantee  (8)  in  Lemma 
1  holds  for  r2;fc.  Then,  (56)  and  (57)  implies  (9)  holds 
for  switching  instants  t2fc+i,  t2k,  respectively.  Finally,  (10) 
can  be  guaranteed  by  (58).  Thus,  according  Lemma  1,  the 
reachable  set  is  obtained  as  1ZX  C  1ZX  =  {a:  £  K"x  | 
xT.S',^+'  <  1  ,q  =  0,1,..., L,i  =  1,2}.  Using  (59),  we 
have  1ZX  C  Rx  =  {x  £  K"x  |  xT Rxx  <  1}.  ■ 

By  adding  the  following  constraint 


Si, 9  -  SI  A  0,  S  >  0,  q  =  0, . . . ,  L,  i  =  1,2  (63) 

The  smallest  ball  13(0,  y/d)  containing  the  reachable  set  TZX 
can  be  obtained  by  the  following  optimization  problem 

min  S  s.t.  (63)  and  (51)  —  (58)  (64) 


Example  1:  Consider  the  plant  described  by 

0  ' 

0 

The  initial  state  is  assumed  to  satisfy  Xo  £  {x$  £  R"  | 
||x0||  <  1},  and  the  control  objective  is  to  ensure  the  state 
trajectories  satisfies  x(t)  £  {x  £  R”  |  ||x||  <  2}.  Assume 
that  the  minimal  reliable  time  for  a  group  of  successfully 
transmitted  information  is  rm in  =  0.5  second,  and  the 
maximal  time  for  a  group  of  successive  packet  dropouts  is 
t/Wx  =  0.1  second.  Let  a  =  0  due  to  Bu  =  [0  0]T,  and 
we  can  find  feasible  solution  to  LMIs  (5 1)— (59)  with  L  =  1. 

Given  an  initial  state  xq  =  [0.6  0.8] T ,  the  state  response  is 
illustrated  in  Fig.  2,  it  can  be  observed  that  the  state  trajectory 
satisfies  x(t)  £  {x  £  R™  |  ||x||  <  2}.  Moreover,  we  generate 
500  random  state  trajectories  with  random  packet  dropouts 
whose  lengths  are  less  than  0.1  second,  it  can  be  seen  that 
all  the  trajectories  are  in  the  prescribed  ball  £>(0,2),  which 
are  shown  in  Fig.  3. 


A  = 


1.5 

1.5 


2.5 

1.2 


By,  = 


0.2 

0.5 


,  By,  = 
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(a)  Occurence  of  packet  dropouts 

(1  indicates  the  normal  working  mode,  2  indicates  the  packet  dropouts) 


Fig.  2.  State  response  of  networked  control  system  with  packet  dropouts 


xf 


Fig.  3.  500  random  state  trajectories  with  random  packet  dropouts.  All  the 
trajectories  x(t)  generated  from  the  | xq |  <  1  are  bounded  by  ||a;(f)||  <  2. 

Finally,  in  order  the  show  how  parameter  L  works  for  the 
controller  design,  different  L  are  selected  for  optimization 
problem  (64).  From  L  =  1  to  L  =  5,  the  smallest  S  are 
computed,  which  are  shown  in  Table  II.  In  Table  II,  we  can 
see  that  S  monotonically  decreases  as  L  increases,  this  is 
consistent  with  Remark  1.  However,  a  selection  of  larger  L 
has  to  afford  more  computational  cost,  the  computation  time 
grows  as  L  increases  in  Table  II. 


TABLE  II 

S  AND  COMPUTATION  TIME  (C.T.)  WITH  L  =  1, 2,  3,  4,  5 


L  =  1 

L  =  2 

II 

CO 

L  =  4 

L  =  5 

s 

1.8795 

1.5075 

1.4615 

1.4434 

1.4334 

C.T. 

0.296  s 

0.433  s 

0.561  s 

0.734  s 

0.905  s 

VI.  Conclusions 

By  employing  a  class  of  time-scheduled  Lyapunov  func¬ 
tions,  the  reachable  estimation  and  control  problems  for 
switched  linear  systems  under  dwell  time  constraint  are 
investigated  in  this  paper.  A  sufficient  condition  has  been 
proposed  to  estimate  the  reachable  set  of  switched  system 
by  bounding  ellipsoids,  then  based  on  the  estimation  result, 
a  time-scheduled  state  feedback  controller  gains  are  obtained, 
which  can  ensure  the  state  trajectories  of  closed-loop  system 
in  a  prescribed  set.  Finally,  the  controller  design  result 


is  applied  into  the  networked  control  system  with  packet 

dropouts. 
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On  Reachable  Set  Estimation  for  Discrete-Time  Switched  Linear 
Systems  under  Arbitrary  Switching 


Weiming  Xiang,  Hoang-Dung  Tran  and  Taylor  T.  Johnson 


Abstract — This  paper  addresses  the  problem  of  reachable 
set  estimation  for  discrete-time  switched  systems  under  arbi¬ 
trary  switching.  By  introducing  a  novel  conception  called  M- 
step  sequence  which  is  capable  of  characterizing  all  possible 
subsystem  activation  orders  during  M  discrete-time  steps,  a 
Lyapunov  function  based  approach  is  proposed  to  derive  a  set  of 
bounding  ellipsoids  to  estimate  the  reachable  set.  The  proposed 
approach  can  cover  the  previous  switched  Lyapunov  function 
approach  and  yields  less  conservativeness.  Moreover,  it  can  be 
shown  that  the  AI -step  sequence  method  can  also  reduce  the 
conservativeness  in  stability  analysis  for  discrete-time  switched 
systems  under  arbitrary  switching  in  contrast  to  switched 
Lyapunov  function  method.  Several  numerical  examples  are 
provided  to  illustrate  our  approach. 

I.  Introduction 

A  switched  system  is  composed  of  a  family  of  continuous 
or  discrete-time  subsystems,  described  by  differential  or 
difference  equations,  respectively,  along  with  a  switching 
rule  governing  the  switching  between  the  subsystems.  The 
motivation  for  studying  such  switched  systems  comes  from 
the  fact  that  switched  system  can  be  efficiently  used  to  model 
many  practical  systems  that  are  inherently  multi-model,  thus 
several  dynamical  subsystem  models  are  required  to  de¬ 
scribe  their  behavior.  For  example,  several  real-world  cyber¬ 
physical  systems  and  industrial  processes  exhibit  switching 
and  hybrid  nature  intrinsically.  Generally,  the  stability  and 
stabilization  problems  are  the  main  concerns  in  the  field  of 
switched  systems,  e.g.,  see  [  1  ]— [4]  and  the  references  cited 
therein.  One  can  study  the  stability  and  other  properties 
of  switched  systems  with  a  given  the  switching  rule  as 
a  prescribed  state  space  partitioning  [5]— [7]  or  with  some 
known  constraints  on  switching  sequence  such  as  dwell 
time  [8]  or  average  dwell  time  [9]  restrictions.  For  instance, 
combining  multiple  Lyapunov  function  (MLF),  the  dwell 
time  and  average  dwell  time  properties  of  relatively  slowly 
switched  systems  have  been  investigated  in  the  correspond¬ 
ing  switched  systems  [10]— [16].  However,  in  a  number  of 
practical  switched  systems,  the  switching  sequence  is  not 
known  a  prior  and  these  properties  have  to  be  examined 
under  arbitrary  switching. 
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the  National  Science  Foundation  (NSF)  under  grant  numbers  CNS  1464311, 
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Reachable  set  estimation  aims  to  derive  a  closed  bounded 
set  that  contains  all  the  state  trajectories  generated  by  a 
dynamic  system  with  a  prescribed  class  of  initial  state  set 
and  inputs.  Reachable  set  estimation  is  not  only  of  theoretical 
interest  in  robust  control  theory  [17],  but  also  closely  related 
to  practical  engineering  for  the  safety  verification  problems 
[18],  For  example,  a  dynamic  system  is  regarded  to  be  safe 
if  its  reachable  set  does  not  intersect  with  the  unsafe  or 
undesirable  sets  of  states.  In  some  early  work,  reachable  set 
bounding  was  considered  in  the  context  of  state  estimation 
and  it  has  later  received  a  lot  of  attention  in  parameter 
estimation,  see  [19]  and  references  therein.  Recently,  em¬ 
ploying  ellipsoidal  techniques  based  on  Lyapunov  function 
approaches  to  estimate  the  reachable  sets  for  different  class 
of  systems  attracts  many  researchers’  attention.  In  the  frame¬ 
work  of  bounding  ellipsoid,  the  quadratic  Lyapunov  function 
has  played  a  fundamental  role  in  the  reachable  set  estimation 
problem,  and  it  has  been  further  extended  and  developed  to 
time-delay  systems  [20]-[22],  singular  systems  [23]. 

For  the  reachable  set  estimation  problem  for  discrete-time 
switched  system  under  arbitrary  switching,  [24]  proposes  a 
method  based  on  switched  Lyapunov  function  approach,  and 
the  trajectories  are  estimated  by  a  set  of  bounding  ellipsoids. 
The  main  aim  in  this  paper  is  to  further  develop  the  Lyapunov 
function  approach  and  reduce  its  conservativeness  in  reach¬ 
able  set  estimation  for  discrete-time  switched  system  under 
arbitrary  switching.  By  introducing  the  conception  of  M-step 
sequence  which  is  able  to  characterize  all  possible  subsystem 
activation  orders  during  M  steps,  an  improved  method  will 
be  proposed  in  this  paper.  It  should  be  stressed  that  the 
approach  in  [24]  can  be  recovered  by  particularly  letting 
M  =  1  and  thus  has  less  conservativeness.  Additionally, 
also  in  virtue  of  the  advantages  of  M-step  sequence,  the  less 
conservativeness  emerges  in  stability  analysis  for  discrete¬ 
time  switched  system  in  comparison  with  the  well-known 
switched  Lyapuonv  function.  Finally,  several  numerical  ex¬ 
amples  are  given  in  order  to  emphasize  the  less  conserva¬ 
tiveness  and  effectiveness  of  the  approach. 

The  remainder  of  this  paper  is  organized  as  follows: 
Preliminaries  and  problem  formulation  are  given  in  Section 
II.  The  main  results  including  the  M-step  sequence,  reach¬ 
able  set  estimation  and  discussion  on  stability  are  given  in 
Section  III.  Numerical  examples  are  provided  in  Section  IV. 
Conclusions  are  given  in  Section  V. 

Notation:  N  represents  the  set  of  natural  numbers.  R.  and 
R>o  denote  the  fields  of  real  numbers  and  nonnegative  real 
numbers,  respectively.  R"  is  the  vector  space  of  all  /(-tuples 
of  real  numbers,  R"x”  is  the  space  ofnxti  matrices  with 
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real  entries.  The  notation  P  >~  0  (P  -<  0)  means  P  is 
real  symmetric  and  positive  definite  (negative  definite).  AT 
denotes  the  transpose  of  A.  In  symmetric  block  matrices, 
we  use  *  as  an  ellipsis  for  the  terms  that  are  introduced 
by  symmetry.  diag{-  •  •  }  denotes  a  block-diagonal  matrix. 
|-|  stands  for  the  Euclidean  norm.  The  bounding  ellipsoid 
is  expressed  by  £{R)  =  {x  £  R"  |  xT Rx  <  1,0  -<  R  € 
R”x"},  and  ball  B{xq,8)  =  {a;  £  1"  |  \x  —  xo|  <  S,x o  £ 
R",  6  >  0}. 

II.  Preliminaries  and  Problem  Formulation 

In  this  paper,  we  consider  a  class  of  discrete-time  switched 
linear  system  in  the  following  form 

x(k  +  1)  =  Aa(k)x{k)  +  Ba(k)u{k)  (1) 

where  x(k),x o  £  Rra*  are  the  state  of  the  system  and 
initial  state,  respectively.  The  switching  signal  er  is  defined 
as  er  :  N  — >  Z[l,  JV],  where  N  is  the  number  of  subsystems 
involved  in  the  switched  system.  In  this  paper,  no  specific 
restriction  is  imposed  on  switching  signal  er,  namely  the 
arbitrary  switching  law  is  considered  in  the  rest  of  paper. 
At,  and  Bi ,  i  £  I[1,N]  are  constant  system  matrices 
with  appropriate  dimensions.  u>(k)  £  Rn"  is  the  bounded 
peak  input  vector  which  is  assumed  to  satisfy  the  following 
constraint 

u(k)  £  W  =  {w  £  R"“  |  wTw  <d2,d>  0}  (2) 


Lemma  2:  [24]  Consider  system  (1)  with  input  (2).  If 

there  exist  matrices  Pi  >-  0,  i  £  I[1,N]  and  scalars  0  < 
ctij  <  1  such  that  V(i,j)  £Z[l,iV]  xl[l ,7V], 


AjPjAj  a 


1,3 


Pi  AJPjB, 

Bj  PjBi  — 


^0  (5) 


then  system  (1)  is  GUAS  and  the  reachable  set  1ZX  can  be 
over  approximated  by  TZX  =  UiGi[i,jv]  £(pi)- 

Remark  1:  In  [24],  Lemma  1  has  Vi (x(k))  <  1,  Vi  £ 
Z[l,  N],  and  the  reachable  set  1ZX  in  Lemma  2  is  bounded 
by  the  intersection  of  a  set  of  ellipsoids  fljczn  n]  £(Pi)- 
We  correct  this  slight  error  as  that  3i  £  Z[l,  N\  such  that 
Vi(x(k))  <  1  and  the  over  approximate  set  1ZX  should  be 
the  union  of  a  set  of  ellipsoids  UiGz[i  n]  £(•?*)>  since  cr(fc) 
is  an  arbitrary  switching  means  cr(fc)  could  be  any  possible 
i  £  L\  1 ,  N]  and  7 Zx  needs  to  include  all  possible  trajectories 
for  any  i  £  Z[l,  N ]. 

On  the  basis  of  above  lemma,  the  over  approximate 
reachable  set  1ZX  can  be  characterized  by  a  set  of  ellipsoids, 
and  optimization  problems  can  be  formulated  to  obtain  1ZX 
as  small  as  possible  in  [24].  In  this  paper,  our  main  aim  is 
to  further  improve  this  Lyapunov  function  based  approach  to 
develop  less  conservative  result  for  reachable  set  estimation 
of  switched  system  (1),  namely  to  develop  an  approach  to 
better  over  approximate  the  reachable  set  1ZX. 


III.  Main  Results 


The  main  problem  considered  in  this  paper  is  the  reachable 
set  estimation  problem  for  switched  system  (1)  with  input 
ut(k)  satisfying  (2).  The  reachable  set  is  defined  as 

1ZX  =  {x  £  R11*  |  xq  =  0 ,x(k),u(k)  satisfy  (1),  (2)}  (3) 

Due  to  the  complex  characteristic  of  switched  systems, 
the  accurate  reachable  set  for  switched  system  (1)  is  hard  to 
compute.  The  reachable  set  estimation  problem  is  formulated 
as  follows. 

Problem  1:  For  switched  system  (1),  determine  an  over 
approximate  set  TZX  such  that  1ZX  C  TZX,  and  the  set  IZX 
should  be  optimized  as  small  as  possible. 

The  recent  solution  to  compute  an  over  approximate 
set  1ZX  is  proposed  in  [24],  which  is  based  on  switched 
Lyapunov  function  approach  [25]. 

Lemma  1:  [24]  Consider  system  (1)  with  input  (2).  If 

there  exist  a  set  of  a  family  of  functions  Vj  :  R"  — >  R+ 
satisfying  V)(0)  =  0  and  Vi(x)  >  0,  Vx  =  0,  i  £  T[1,N], 
and  exist  scalars  0  <  a,  j  <  1  such  that  V(i,  j)  £  Z[l,  N]  x 

Vj(x(k  +  1))  -  otijVi(x(k))  -  ujt (k)u(k)  <  0 

(4) 

then  system  (1)  is  globally  uniformly  asymptotically  stable 
and  we  have  3i  £  Z[l,i\T]  such  that  Vi(x(k))  <  1  for  all 
x(0)  satisfying  V)(x(0))  <  1,  Vi  £  I[1,N], 

In  the  framework  of  quadratic  switched  Lyapunov  func¬ 
tion,  the  following  result  for  reachable  set  estimation  stems 
from  above  Lemma. 


In  this  section,  the  reachable  set  estimation  problem  will 
be  studied  based  on  a  novel  conception  named  M -step  se¬ 
quence,  an  LMI  based  approach  will  be  proposed  to  obtain  a 
set  of  bounding  ellipsoids.  Moreover,  the  globally  uniformly 
asymptotical  stability  of  discrete-time  switched  linear  system 
is  discussed  in  the  framework  of  M-step  sequence. 

A.  M-Step  Sequence 

In  this  paper,  the  main  aim  is  to  further  reduce  the 
conservatism  in  Lyapunov  function  based  approach  for  reach¬ 
able  set  estimation  of  discrete-time  switched  system  over 
switched  Lyapunov  function  methods.  First,  we  introduce 
the  conception  of  the  M- step  sequence,  which  plays  a 
fundamental  role  in  this  paper.  The  M-step  sequence  is 
defined  as  follows. 

Definition  1:  For  a  switched  system  consisting  N  subsys¬ 
tem,  and  given  a  time  window  with  M- step  length,  an  M- 
step  sequence  is  a  combination  of  subsystems  in  M  steps. 
There  are  NM  combinations  of  subsystems  in  M  steps,  and 
these  Nm  combinations  are  indexed  by  T[1,NM\.  For  the 
ith  sequence  of  combination  in  Z[l,  NM],  it  is  expressed  by 

sf  =  {*1,  *2,  ■  •  •  ,  *m},  *!,  •  •  •  ,  *M  £  Z[l,  N],  i  £  2[1,  Nm] 

The  M-step  sequence  is  able  to  characterize  all  possible 
activation  orders  for  switched  system  during  the  M  steps. 
Given  a  switching  signal  a(k)  in  [0,  oo),  we  denote  the  ?7th 
M-step  activation  sequence  is 

=  {cr{nM),a{nM  +  1), . . . ,  <j((n  +  1)M  —  1)}  (6) 

where  n  =  0, 1, . . .. 
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The  following  properties  can  be  easily  observed. 

Proposition  1:  Given  any  switching  signal  er(fc)  defined 
over  interval  [0,  oo),  we  have 

1)  U:=oSn  =  M0)^(1),a(2),...}. 

2)  For  any  n  =  0,1,...,  there  exists  an  i  £  X[1,NM ] 
such  that  E n  =  Sf1 . 

Remark  2:  The  first  property  implies  the  activation  order 
of  any  switching  signal  a(k)  can  be  expressed  by  Af-step 
activation  sequence  E„  as  n  — »  oo.  The  second  property 
means  any  M-step  activation  sequence  E„  can  be  found  in 
SAI ,  i  £  T[1,NM],  These  two  properties  suffice  to  show 
that  the  M-step  sequence  SAI ,  i  £  X[1  ,NM],  is  capable  to 
describe  the  behaviors  of  switching  signal  cr(k)  in  [0,  oo). 

Based  on  the  introduced  notion  of  M-step  sequence, 
we  introduce  a  class  of  A/ -step  clock-dependent  switched 
Lyapunov  functions  V)  :  [nM  +  1,  (n  +  1)M]  x  R"  — ►  M+, 
n  =  0,1,...,  i  S  I[l,  Nm],  associated  to  Af-step  sequence, 
which  are  a  family  of  non-negative  functions  satisfying 

Pi(\x\)  <Vi(k,x)  <  (32(\x\)  (7) 

where  /3i,/32  £ 

In  the  framework  of  the  A/-step  clock-dependent  switched 
Lyapunov  function,  the  following  result  can  be  obtained  as 
an  improvement  of  Lemma  1 . 

Theorem  1:  Consider  system  (1)  with  input  (2).  If  there 
exist  a  set  of  a  family  of  non-negative  functions  F)  :  [nM  + 
1,  (n  +  1)M]  x  Rn  -►  K+,  n  =  0, 1, . . .,  i  £  I[l,  NM] 
satisfying  (7),  and  exist  scalars  0  <  a{.  aiij  <  1  such  that 
V(i,j)  £l[l,NM]xI[l,NM], 

<  0,  Vfc  =  nM+l,...,(n+  1)M  (8) 

B itj  <  0  (9) 

where  f \(k)  =  Vi {k  +  1  ,x(k  +  1))  —  aiVi(k,x(k))  — 
(k)ui{k),  0,; j  =  Vj(nM  +  1  ,x(nM  +  1))  — 
aijVi(nM,  x(nM))—  1  J;*’3  uT  (nM)u>(nM),  n  =  1,2,.... 
Then  system  (1)  is  uniformly  stable  and  we  have  3i  £ 
X[1,NM]  such  that  Vi(x(k))  <  1  for  all  Xo  satisfying 
Vi(0,xo)  <  1,  Vi  £  X[1,NM]. 

Proof:  First,  we  consider  u(k)  =  0  for  stability.  By  (8), 
it  ensures  that 


Furthermore,  in  presence  of  input  u>{k),  (8)  yields  that 

Vi {k  +  1  ,x(k  +  1))  -  aiiVi(k,x(k))  <  Ql '  uT (fc)cu(fc) 

az 

<  1  -  Oti 


(13) 


which  implies  Vi(k  + 1,  x(fc  + 1))  —  1  <  a* (Vj(fe,  x(k))  —  1). 

Similarly,  (9)  can  lead  to 

Vj(nM  +  1,  x(nM  +  1))  —  aiijVi(nM ,  x(nM ))  <  1  —  a,; 

(14) 


holds  for  n  =  1,2,...,  which  implies 


Vj(nM  +  l,x(nM  +  1))  —  1  <  aiij(Vi(nM,x(nM))  —  1) 

(15) 

For  any  k  £  N,  we  have 

V„ (k)(k)  —  1 

i)  (Va(k— i)  (k  1)  1) 

—  1)  '  '  '  rKcr(nAr  +  l)  (1 f(nM)  ( u A7  -(-1)  1) 

—  —  1)  '  '  '  rT(j(7iiVf+l)  ,a(nM)  (Va(nM)  (uA/)  1) 

—  —  1)  '  '  '  rKcr(nAT) ,cr(n  M  —  1)  '  '  '  0^(0)  (LrtO)  (0)  1) 

Due  to  0  <  a,, a.jj  <  1  and  V)(0,xo)  <  1,  V«  £ 
X[1,NM],  it  ensures  V^^fk)  —  1  <  0,  Vfc  £  N.  Because 
o(k)  is  an  arbitrary  signal  selecting  value  in  X[1,NM],  it 
implies  3i  £  X[1,NM]  =>  V)(fc,  x)  <1.  ■ 

Remark  3:  If  we  particularly  let  M  =  1,  Condition  (8)  is 
reduced  to 

Vi(n+l,x(n+l))—aiVi(n,x(n))~  1  J*1  uT (ri)oj(n)  <  0 

a 

(16) 

and  (9)  can  be  rewritten  to 

Vj-(n+l,x(n  +  l))  —  aijVi(n,  x(n))  —  - — ^^-u;T(n)cu(n)  <  0 

(17) 

It  is  noted  that  (16)  can  be  absorbed  in  (17)  by  just  letting 
o.i,i  =  on.  It  can  be  seen  that  (17)  is  exactly  the  condition  (4) 
in  Lemma  1.  Therefore,  Theorem  1  covers  previous  result 
stated  by  Lemma  1,  namely  Lemma  1  is  a  particular  case 
which  can  be  recovered  by  Theorem  1  with  M  =  1. 


B.  Reachable  Set  Estimation 


Vi{k  +  1,  x(k  +  1))  —  aiV,(k,  x(k))  <  0  (10) 

holds  for  k  =  nM  +  1,  nM  +  1, . . . ,  (n  +  1  )M. 

Then,  by  (9),  one  has 

Vj(nM  +  1,  x{nM  +  1))  -  aidVi(nM ,  x(nM))  <0  (11) 

Define  a  new  function  a  :  N  — >  X[1,NAI]  indicating  the 
active  AT-step  sequence,  and  choose  a  Lyapunov  function 
candidate  as  Vg (k)(k,x(k)).  According  to  Proposition  1,  and 
together  with  (10)  and  (11)  with  the  fact  0  <  a.;,  altj  <  1, 
it  leads  to 

VHk+1){k  +  1  ,x(k  +  1))  -  Va^(k,x(k))  <  0  (12) 

Combined  with  (7),  the  stability  can  be  established  by 
standard  Lyapunov  theorem. 


In  this  subsection,  the  reachable  set  estimation  for  discrete¬ 
time  switched  linear  system  will  be  investigated.  Based  on 
Theorem  1,  the  following  result  can  be  obtained. 

Theorem  2:  Consider  system  (1)  with  input  (2).  If  there 
exist  matrices  Pjjm  >-  0,  m  £  X[1,M],  i  £  X[1,NM]  and 
scalars  0  <  a*  <  1,  0  <  aij  <  1  such  that  V(i,j)  £ 
X[1,Nm]  x  X[1,Nm], 


AimPi,m+lAim  (kj  Pjr 
* 


A  T  p  p . 

^ im 1 

pT  p  p  _  1  —  Oti 

d 2 


I 


A  0 


M 


m  =  1, 2, . . . ,  M  —  1 
(18) 

■T  p  p  _ 

iMrP1JD  'M  d2 


A  0 
(19) 
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then  system  (1)  is  uniformly  stable  and  the  reachable  set  1ZX 
can  be  over  approximated  by 

nx  =  U  £(Pi,m)  (20) 

meI[l,M],ieI[l,JVM] 

Proof:  Choosing  an  M- step  clock-dependent  switched 
Lyapunov  function  in  the  following  quadratic  form 

VS(k){k,x{k))  =  XT  (k)Pfr(k),k-nM+lx(k),  U  =  0,1,  .  .  . 

(21) 

where  a(k)  indicating  the  active  M-step  sequence  defined 
same  as  in  (12). 

Suppose  a(k)  =  i,  k  G  [ nM  +  1,  (n  +  1  )M]  and 
denote  flj(fc)  =  V) (k  +  1  ,x(k  +  1))  —  ctj Vi{k,x(k))  — 
1~^fiojT(k)uj(k),  and  thus  the  M-step  sequence  SN  = 
{i\,  *2, . . . ,  %m}-  Along  the  system  evolution,  we  can  obtain 

fli(k)  =ZT(k)Zi,mt(k) 

where  m  G  I[1,M],  £(fc)  =  [xT  (k) ,  coT  (k)]T  and  H*  m  = 
Xz.m-f-l  Ajm  OtiPi^m  Pirn  P^m 

BjmPi,mBim  -  • 

Moreover,  assume  cr(nM)  =  i  and  a(nM  +  1)  =  j,  and 
let  0j j  =  Vj(nM  +  l,x(nM  +  l))—aijVi(nM,x(nM))  — 
1~S*,J  ut1  (nM )ui(nM),  the  following  derivation  can  be  ob¬ 
tained  for  the  transition  from  instant  nM  to  nM  +  1. 

=  (T(nM)n,^(riM) 


TABLE  I 

Computational  complexity  of  Theorem  2 


Number  of  variables 

Size  of  LMIs 

Theorem  2 

n(n+l)MNM 

_ 2 _ 

n(N2M  +  MNm ) 

13(0, 1/ yfe),  Vk  G  K>0,  so  we  have  to  maximize  e  to  obtain 
a  smallest  ball  13(0, 1/s/e)  as 

max  e 

(23) 

s.t.  (18),  (19)  and  (22) 

Moreover,  due  to  the  existence  of  tuning  parameters  cti  and 
atij,  the  result  in  Theorem  2  and  corresponding  optimization 
problem  (23)  are  not  standard  LMI  problems,  they  are 
bilinear  matrix  inequality  (BMI)  problems  and  known  to 
be  NP-hard.  Fortunately,  several  algorithms  are  available 
to  solve  BMI  problems  such  as  the  iterative  linear  matrix 
inequality  (ILMI)  approach  in  [26],  [27],  or  using  numerical 
optimization  algorithms,  such  as  program  fminsearch 
[20]  or  genetic  algorithm  (GA)  [24]  in  the  optimization 
toolbox  of  Matlab. 

Remark  6:  Although  M  >  1  will  reduce  the  conservative¬ 
ness,  the  price  to  pay  is  the  increase  of  computational  com¬ 
plexity.  The  number  of  LMIs  and  involved  decision  variables 
grows  as  M  is  increased.  The  computation  complexities  are 
listed  in  Table  I. 


where 


n  i,j  — 


A 


B 


A. 
tPx  1 


Pj  i 1  B^m 

BiM  -  1-^I 


M 


~1P~ 


By  (18)  and  (19),  it  can  be  ensured  that  f \(k)  <  0,  Vfc  = 
nM  +  1, . . . ,  (n  +  1  )M,  \/n  =1,2,...  and  (~),  j  <  0. 

According  to  Theorem  1,  for  the  case  of  xq  =  0,  we 
have  3i  G  X[l,iVM]  such  that  Vfxfk))  <  1.  Therefore, 
the  state  x(k)  satisfies  x  G  {x  |  xT Pl  rnx  <  1  ,m  G 

X[l,M\,i  G  T[1,NM]}  =  Um£l[l,M],ieI[l,IVM]  £{Pi,m)t 

which  is  exactly  the  set  (20).  The  proof  is  complete.  ■ 
Remark  4:  Theorem  2  can  be  viewed  as  an  improved 
version  for  Lemma  2,  if  we  enforce  M  =  1  in  Theorem  2, 
Pi,m,  m  G  J[l,  M],  i  G  X[l,  NM],  becomes  Pj,  i  G  T[l,  IV] . 
Inequalities  (18)  and  (19)  can  be  rewritten  to 


Aj  PjAi  -  aijPi 


AjPjBi 


A  o 


which  is  (5)  in  Lemma  2. 

Remark  5:  The  set  1ZX  is  usually  expected  to  be  as  small 
as  possible  to  achieve  a  precise  estimate  of  reachable  set  lZy. 
In  [24],  several  methods  have  been  proposed  to  minimize 
the  bounding  ellipsoids,  which  can  be  also  employed  in  our 
paper.  In  order  to  make  a  clear  comparison  with  [24],  we 
consider  the  method  associated  to  the  following  constraint 


Pi,m  h  el,  e>  0,  Vm  G  I[l,  M\,  Vi  G  X[  1,  NM }  (22) 

which  implies  that  exT  (k)x(k)  <  xT  (t)Pi  mx(k)  <  1, 
namely  x(t)  G  XLx  =  Um6i[ijM],i£i[i,ivM]  £{Pi,m)  Q 


C.  Some  Discussions  for  Stability  Analysis 

It  should  be  noted  that  the  stability  analysis  result  of 
switched  system  (1)  with  input  uj(k)  =  0  is  actually 
included  in  the  previous  reachable  set  estimation  solution. 
As  what  has  been  shown  in  previous  section,  our  reachable 
set  estimation  yields  less  conservativeness  than  that  in  [24] 
which  is  essentially  based  on  switched  Lyapunov  function 
approach  in  [25]  .  In  fact,  by  introducing  the  concept  of  M- 
step  sequence,  a  less  conservative  stability  analysis  result  can 
be  obtained  as  well  in  contrast  to  the  well  known  stability 
criterion  proposed  in  [25]  on  basis  of  switched  Lyapunov 
function  approach. 

The  following  corollary  can  be  viewed  as  an  improvement 
for  the  classical  switched  Lyapunov  function  approach  in 
stability  analysis. 

Corollary  1:  Consider  switched  system  (1)  with  ix(k)  = 
0,  if  there  exist  MNm  symmetric  matrices  Pjj?ra  >-  0,  m  G 
X[0,  M\,  i  G  I[l,  NM]  such  that  the  following  inequalities 
hold  for  Vi,j  G  X[1,NM],  Vto  G  X[1,M], 

AJm+1Pi,m+iAim  -  Pi,m  A0,  TO  =  1, 2, . . . ,  M  -  1  (24) 
AJMBj,iAiM  ~  P%,m  A  0  (25) 

then  switched  system  (1)  is  globally  uniformly  asymptotical¬ 
ly  stable. 

Proof:  The  proof  can  be  obtained  by  the  guidelines  in 
Theorems  1  and  2,  which  is  omitted  here.  ■ 

Remark  7:  Corollary  1  can  be  viewed  as  an  improved  re¬ 
sult  over  switched  Lyapunov  function  approach  for  switched 
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system  (1).  By  letting  M  =  1,  conditions  (24)  and  (25)  can 
be  rewritten  to 

Aj P3Az  -  Pt  ~<  0,  i,j  G  Z[  1,  N\  (26) 

where  Pi  y  0,  i  G  X[l,  iV],  This  result  is  exactly  the 
Theorem  2  in  [25],  which  means  that  the  switched  Lyapunov 
function  approach  is  a  special  case  of  Corollary  1  as  M  =  1. 
Corollary  1  with  M  >  2  is  able  to  yield  less  conservativeness 
in  stability  analysis,  which  can  be  shown  by  a  numerical 
example  later. 


IV.  Example 


Example  1:  Consider  a  switched  system  with  two  modes 
with  the  following  system  matrices 


A\ 

Ai 


0 

0.7 

,  B[  = 

0.2 

-0.2 

-0.6 

-0.4 

-0.6 

0.4  ' 

b2  = 

-0.6  ' 

-0.7 

0.2 

5 

0.4 

The  disturbance  uj{k)  satisfies  ui{k)  G  W  =  (w  G  R"™  | 
oj  1  uj  <  1}.  In  order  to  compare  our  approach  with  that 
in  [24],  we  first  use  Lemma  2  to  obtain  the  reachable  set 
estimation  by  maximizing  e  in  optimization  (23).  The  GA  is 
used  to  search  for  optimized  at,  i  G  X[l,  2],  The  population  is 
set  to  be  50.  After  100  generations,  the  optimal  e  =  0.04057, 
which  is  shown  in  Fig.  1. 

On  the  other  hand,  with  same  population  and  generation. 
Theorem  2  with  M  =  2  reaches  a  larger  e  as  e  =  0.05618, 
which  obviously  is  a  less  conservative  result.  The  update  of  e 
at  each  generation  is  illustrated  in  Fig.  1,  which  has  a  slower 
convergent  rate  but  a  better  optimized  result.  The  slower 
convergent  rate  is  basically  because  more  variables  ctjj, 
i,j  G  T.\  1 , 4],  are  introduced  in  the  optimization  problem. 
The  union  of  bounding  ellipsoids  are  depicted  in  Fig.  2  by 
solid  blue  lines.  For  the  purpose  of  showing  the  advantage  of 
our  approach,  we  present  Fig.  3  to  clearly  compare  Theorem 
2  and  Lemma  2,  in  which  the  estimation  by  Theorem  2  is 
more  precise  than  by  Lemma  2.  In  Figs.  2  and  3,  the  state 
trajectories  are  generated  with  arbitrary  switching  signal  and 
disturbance  w(fc)  uniformly  distributed  over  [—1,1]. 

Example  2:  In  this  example,  we  will  show  the  less  conser¬ 
vativeness  of  M- step  method  in  the  stability  point  of  view. 
Let  us  consider  the  system  (1)  with  matrices  Ai  =  eBiT, 
where 


Bi 


0  1 
-10  -1 


B2 


0  1 
-0.1  -4 


(27) 


Letting  T  =  0.1,  and  using  switched  Lyapunov  function 
approach  in  [25]  (also  viewed  as  M  =  1  in  our  M-step 
sequence  approach),  it  can  be  found  that  the  LMI  problem 
is  not  feasible,  so  that  the  globally  uniformly  asymptotically 
stability  cannot  be  determined  by  the  approach  in  [25]. 
Moreover,  by  applying  the  method  in  [28],  the  minimum 
admissible  dwell  time  is  computed  as  2,  which  also  in¬ 
dicates  that  the  globally  uniformly  asymptotically  stability 
of  switched  system  (1)  cannot  be  ascertained  for  the  case 


Fig.  1.  Fitness  function  value  along  with  generations. 


Fig.  2.  Bounding  ellipsoids  by  Theorem  2. 


X1 


Fig.  3.  Bounding  circles  by  Lemma  2  and  Theorem  2. 
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Fig.  4.  State  response  under  switching  occurring  at  each  time  instant. 


of  arbitrary  switching,  for  which  the  minimum  dwell  time 
should  be  1. 

However,  if  we  increase  M  by  just  letting  M  =  2 
in  the  M-step  sequence  method  proposed  in  this  paper, 
the  feasibility  of  the  corresponding  LMI  problems  can  be 
established,  which  is  sufficient  to  guarantee  that  the  system 
is  globally  uniformly  asymptotically  stable  under  arbitrary 
switching.  The  convergent  state  evolution  is  shown  by  the 
following  simulation  result  in  Fig.  4,  where  the  extreme 
switching  behavior,  i.e.,  the  switching  occurs  at  each  time 
instant,  is  adopted,  and  the  initial  state  is  assumed  to  be 
x0  =  [3  5]T. 

V.  Conclusions 

The  reachable  set  estimation  problem  for  discrete-time 
switched  system  has  been  investigated  in  this  paper.  A 
novel  conception  called  M-step  sequence  is  introduced  to 
solve  the  reachable  set  estimation  problem,  it  is  shown  that 
the  proposed  approach  covers  the  previous  result  which  is 
based  on  switched  Lyapunov  function,  and  thus  has  less 
conservativeness.  In  addition,  some  discussions  are  given  for 
stability  analysis  for  discrete-time  switched  system  in  the 
framework  of  M-step  sequence.  Finally,  numerical  examples 
are  given  to  show  the  theoretical  findings  in  this  paper. 
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Event- Triggered  Control  for  Continuous-Time 
Switched  Linear  Systems 
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Abstract 

The  event-triggered  control  problem  for  switched  linear  system  is  addressed  in  this  paper.  The 
periodical  sampling  scheme  and  event-triggering  condition  are  incorporated  in  the  closed-loop.  The 
feedback  control  updates  its  value  only  at  sampling  instants  as  long  as  event-triggering  condition 
is  satisfied  as  well.  In  addition,  the  switchings  are  only  allowed  to  occur  at  sampling  instants  and 
meanwhile  the  switching  condition  is  satisfied.  Three  equivalent  sufficient  conditions  are  proposed 
to  ensure  the  asymptotic  stability  of  switched  systems.  In  particular,  one  condition  has  a  promising 
feature  of  affineness  in  system  matrices,  and  as  a  consequence,  it  is  extended  to  robust  sampling 
case  and  ^2-gain  analysis.  Several  examples  are  provided  to  illustrate  our  results. 

Keywords:  Asymptotic  stability,  event-triggered  control,  £2  gain,  switched  systems 


1  Introduction 

Switched  systems  have  emerged  as  an  important  subclass  of  hybrid  systems  and  represent  a  very  active 
area  of  current  research  in  the  field  of  control  systems  [ffl-0j.  A  switched  system  is  composed  of  a  family 
of  continuous  or  discrete-time  subsystems,  described  by  differential  or  difference  equations,  respectively, 
along  with  a  switching  rule  governing  the  switching  amongst  the  subsystems.  The  motivation  for 
studying  switched  systems  comes  from  the  fact  that  switched  system  can  be  effectively  used  to  model 
many  practical  systems  that  are  inherently  multi-model  in  the  sense  that  several  dynamic  subsystem 
models  are  required  to  describe  their  behavior.  For  instance,  the  sampled  data  systems  [0] ,  networked 
control  systems  [0]  and  event-triggered  systems  [0]  can  be  modeled  as  switched  systems.  Generally,  the 
stability  and  stabilization  problems  are  the  main  concerns  in  the  field  of  switched  systems.  It  has  been 
proved  that  Lyapunov  function  techniques  are  effective  to  deal  with  stability  and  stabilization  problems 
for  switched  systems,  for  example  [ta-0].  Combining  multiple  Lyapunov  function  (MLF),  the  dwell  time 
and  average  dwell  time  properties  of  relatively  slowly  switched  systems  have  been  investigated  in  the 
corresponding  switched  systems  [inm!].  For  more  details  on  the  recent  advances  in  the  area,  the 
readers  are  referred  to  the  surveys  [0],  and  the  references  cited  therein. 

On  the  other  hand,  the  periodic  and  aperiodic  control  strategies  are  presented  as  the  most  prevailing 
control  approaches  on  digital  platforms.  Typically,  the  control  executes  periodically  in  the  closed-loop 
and  the  system  can  be  analyzed  by  the  well-developed  sampled-data  system  theory.  As  a  further 

f  Authors  are  with  the  Department  of  Electrical  Engineering  and  Computer  Science,  Vanderbilt  University,  Nashville, 
TN  37212  USA.  Email:  Weiming  Xiang  (xiangwming@gmail.com),  Taylor  T.  Johnson  (taylor.johnson@gmail.com). 
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improvement  of  traditional  sampled-data  system,  the  event-triggered  control  system  is  introduced,  see 
for  example  the  theory  work  [o-aa],  and  numerous  applications  [O-EO].  In  the  framework  of  event- 
triggered  control,  the  control  executions  are  generated  by  well-designed  event-triggering  condition.  In 
comparison  with  sampled-data  scheme,  the  event-triggered  control  which  is  a  typical  aperiodic  one  is 
capable  of  significantly  reducing  the  number  of  control  task  executions,  while  retaining  a  satisfactory 
closed-loop  performance.  Though  the  event-triggered  control  can  offers  some  clear  advantages  with 
respect  to  periodic  control  such  as  in  handling  energy,  computation,  and  communication  constraints 
but  it  also  introduces  some  new  theoretical  and  practical  problems.  The  detailed  advantages  and 
challenges  introduced  by  the  event-triggered  control  can  be  found  in  the  survey  paper  [^11- 

In  this  paper,  we  consider  a  class  of  periodic  event-triggered  control  for  switched  linear  systems. 
The  periodic  event-triggering  condition  allows  the  coexistence  of  periodic  sampling  scheme  and  event¬ 
triggering  condition  for  the  control  executions.  Moreover,  this  blending  strategy  also  determines  the 
occurrence  of  switching  behaviors,  in  other  words,  the  switching  only  occurs  at  sampling  instants  as 
long  as  the  switching  condition  is  satisfied.  Three  stability  criteria  are  proposed  for  event-triggered 
switched  system  in  this  paper,  and  they  are  proved  to  be  basically  equivalent.  The  first  one  is  derived 
by  analyzing  the  evolution  of  state  at  sampling  instant,  however,  it  is  not  convenient  to  extend  to 
further  problems  such  as  robust  sampling  and  £2-gain  analysis.  Then,  a  sampling-dependent  approach 
is  proposed,  which  actually  is  not  numerically  tractable  since  it  has  infinitely  many  values  to  check. 
Thus,  a  discretized  method  to  equivalently  convert  the  sampling-dependent  condition  into  a  numerically 
tractable  condition.  Based  on  this  numerically  tractable  condition,  the  extensions  to  robust  sampling 
case  and  £2-gahi  analysis  are  made. 

The  remainder  of  this  paper  is  organized  as  follows:  The  event-triggered  switched  system  model 
is  given  in  Section  2.  The  main  result,  three  equivalent  stability  criteria  are  presented  in  Section 
3.  Extensions  to  robust  sampling  case  and  £2-gain  analysis  are  studied  in  Section  4  and  Section  5, 
respectively.  Conclusions  are  given  Section  6. 

Notations:  N  represents  the  set  of  natural  numbers,  R.  denotes  the  held  of  real  numbers,  R+  is  the 
set  of  nonnegative  real  numbers,  and  R”  stands  for  the  vector  space  of  all  n-tuples  of  real  numbers, 
R™xn  is  the  space  ofnxn  matrices  with  real  entries.  The  set  M”  consists  of  all  matrices  £  R”xrl 
with  nonnegative  off  diagonal  elements  <pji  >  0,  i  ^  j,  satisfying  y~h_  1  4>jj  =  0,  which  implies  that 
4>u  <  0-  The  set  consists  of  all  matrices  II  £  Rraxn  with  nonnegative  elements  7 >  0  satisfying 
the  normalization  constraints  X^=i  T?'*  =  T  INI  stands  for  Euclidean  norm.  The  notation  A  >-  0 
means  A  is  real  symmetric  and  positive  definite.  A  >-  B  means  that  A  —  B  >~  0.  AT  denotes  the 
transpose  of  A.  In  addition,  in  symmetric  block  matrices,  we  use  *  as  an  ellipsis  for  the  terms  that  are 
induced  by  symmetry  and  diag{-  •  •  }  stands  for  a  block-diagonal  matrix.  /  denotes  the  unit  matrix  and 
0  stands  for  the  zero  elements  in  matrix  with  appropriate  dimensions.  We  define  x(t ^ )  =  limt_^+  x(t) 
and  x(t jjT)  =  lim^j-  x(t).  For  a  matrix  function  F  :  [a,b\  — >  R”xrl,  its  upper  right  Dini  derivative  is 

defined  by  V+F(x )  =  lim?l_>cl+  sup  FU+hPF<-x)  _  jn  ^he  rest  0f  this  wor]C)  we  wiH  make  extensive  uses 
of  the  following  matrix  expressions: 

tf(A,  P)  =  AtPt  +  PA 
®{A,  P(t ))  =  V{A,  P{t))  +  V+P(t) 

®i(A,  P,  Q,  S)  =  V(A,  P)  +  (P  -  Q)/S 
$>2(A,  P,  Q,  S )  =  If  (A,  Q)  +  (P-  Q)/S 
g(A,  J,  P,  Q,  t)  =  eATtJTPJeAt  -  Q 
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2  Event-Triggered  Switched  Control  System 

Consider  the  continuous-time  switched  linear  system  in  the  following  form: 


x(t)  =  Aa{t)x(t)  +  Ba{t)u(t)  +  Ea{t)ui(t)  (1) 

y{t)  =  Ca(t)x(t)  +  Da[t)u){t)  (2) 


where  x{t),x o  £  K"  are  the  state  of  the  system  and  the  initial  condition,  respectively.  u(t)  £  K"“  is 
the  input  and  u(t)  £  K"1"  is  the  exogenous  disturbance.  y(t)  £  Mn*  is  the  controlled  output.  The 
switching  function  a  :  M+  — J\f  =  {1,  2, . . . ,  N}  defines  the  switching  actions,  where  N  is  the  number 
of  subsystems. 

In  this  paper,  we  consider  a  periodic  event-triggered  control  strategy  for  switched  system  (□)-(□)  for 
the  sake  of  taking  advantages  of  both  periodic  sampled-data  and  event-triggered  control,  which  means 
the  system  state  x(t)  is  only  measured  at  the  periodic  sampling  times  for  generating  the  control  input, 
computing  the  switching  function  output  and  verifying  the  event-triggering  condition.  In  a  periodic 
sampling  implementation,  the  values  of  the  system  state  are  available  for  a  time  sequence  S  =  {tfejfcgN, 
where  to  is  the  initial  time  and  tk,  k  £  N\  {0},  are  the  sampling  times,  which  are  periodic  in  the  sense 
that  tk  =  kTs ,  k  £  N,  for  some  properly  chosen  sampling  interval  Ts  >  0.  With  this  sampling  setting, 
the  sampled  switching  signal  is 

a(t)  =  a(t),  t  £  {tk,tk+ 1]  (3) 

where  a(t),  t  £  (tk,tk+ i],  is  determined  by 


f  cr (tk)  cr(4)  /  a(tk) 
l  a{tk)  =  critk) 


(4) 


The  sampled  switching  signal  (0)-(0)  implies  the  switching  decisions  are  only  made  at  sampling 
instant  tk-  The  value  of  a(t)  only  changes  at  sampling  instant  tk  if  <j{tk)  ^  d(tfc),  otherwise  it  holds 
its  most  recent  value.  It  worth  mentioning  that  since  the  switching  function  (0)  only  activates  at  each 
sampling  time  tk,  k  £  N,  it  can  be  interpreted  that  a  dwell  time  constraint  tk+i  —  tk  >  Ts,  \/k  £  N  is 
imposed  on  the  switching  signal.  This  dwell  time  constraint  obviously  prevents  the  switching  actions 
from  chattering  phenomenon  or  Zeno  phenomenon,  since  the  switching  frequency  is  restricted  to  have 
an  upper  bound  equals  to  1/TS.  In  [E2],  a  modified  min-switching  law  with  dwell  time  constraint 
is  proposed  to  avoid  the  chattering  behavior  owe  to  the  dwell  time  constraint.  However,  it  requires 
accessing  the  system  state  and  monitoring  the  state-dependent  switching  rule  continuously,  which  is 
not  allowed  in  the  sampled-data  setting  proposed  in  this  paper,  since  the  system  state  x(t)  is  obtained 
only  at  sampling  instants. 

In  addition,  we  also  take  the  sampled-data  feedback  controller  into  account.  In  a  conventional 
periodic  sampled-data  control  scheme,  the  following  mode-dependent  state  feedback  controller  is  often 
considered 

u{t )  =  Ka^x(t),  t£  R+  (5) 

where  Ki:  i  £  J\f  are  the  already  designed  feedback  gains  for  subsystems,  and  x(t),  t  £  (tk,tk+  i],  is 
defined  by 

x(t)  =  x{tk),  t  £  (tk,tk+ i]  (6) 
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Condition  r(x(t,)) 

Sampling 
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Figure  1:  General  scheme  of  periodic  event-triggered  switched  control  system 


In  order  to  obtain  a  complete  model  of  system  (□)  (□)  with  the  periodic  sampling  setting  (□)  and 
(0),  we  let  x(t )  =  [ x{t )  a:(f)]T  and  obtain  the  following  system 


x{t)  =  Aa{t)x(t)  +  EaWui(t),  t  £  R+  \  S 
x{tt)  =  Jx(tk),  tk&S 
y(t )  =  Ca(t)x(t)  +  Da(t)u(t) 

where  a(t)  evolves  according  to  (□)  and 


Ai  BiKi 

'  Ei  ' 

,  J  = 

'  I  O' 

0  0 

,  Ei  = 

0 

JO 

Ci  =  [Ci  0 


£>i  =  Di 


(7) 

(8) 
(9) 


Further  considering  the  event-triggered  controller,  the  state  measurements  are  transmitted  over  a 
communication  network  and  the  control  values  are  updated  only  when  certain  event-triggering  condi¬ 
tions  are  satisfied,  the  controller  is  given  in  the  following  form 


u{t)  =  Ka(t)x(t),  t  €  K+ 

where  x(t)  is  a  left-continuous  signal,  given  for  t  £  {t]iltk+ i],  k  £  N,  and  modifies  the  (0)  as 


(10) 


f  x(tk),  T(x(tk),x(tk))  >  o 
l  x(tk),  T(x(tk),x(tk))  <  0 


with  an  event-triggering  function  F  :  M2n  — >  K.  The  value  x(tk)  stands  for  the  valid  value  for  the 
controller  at  sampling  time  tk  and  through  the  successive  interval  [tk,tk+ 1),  which  is  determined  by 
the  event-triggering  function  F.  If  T(x(tk),x(tk))  <  0,  the  state  x(tk)  holds  as  its  most  recent  value, 
and  in  the  case  of  T(x(tk),  x(tk))  >  0,  the  state  x(tk)  is  transmitted  over  the  network  to  the  controller 
and  x(tk)  is  updated  accordingly.  The  general  scheme  of  event-triggered  switched  control  system  with 
periodic  sampling  setting  is  illustrated  in  Figure  □. 

In  this  paper,  we  focus  on  a  class  of  quadratic  event-triggering  condition,  that  is,  T(x(tk),x(tk))  is 
in  the  following  quadratic  form 

r(x(ifc))  =  xT(tk)Qx{tk)  (12) 
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where  x(tk)  =  [xT  (tk)  iT(tfe)]T  and  Q  £  M2 nx2n  jg  a  Symmetric  matrix.  Several  event-triggering 
conditions  can  be  written  into  the  quadratic  structure  (O) ,  for  example  the  state-error  based  triggering 
condition  T(x(tk),x(tk))  =  || x(tk)  —  x(tfc)||  —  A  || cc(tfe) || ,  where  A  >  0,  can  be  expressed  by  (O)  with 

f  (1  -  A2)/  -/  ' 


Other  well-known  event  triggering  conditions  such  as  input-error  based,  Lyapunov  function  based 
conditions  can  be  formalized  by  (O)  as  well,  readers  can  refer  to  [0]. 

In  summary,  by  modifying  the  periodic  sampled-data  system  model  (□)-(□),  the  event-triggered 
system  model  arrives  at 


x(t)  =  AaWx(t)  - 

y(t)  =  caWx(t)  - 


■  Ea{t)0j{t),  t  £  M+  \  S 
xT (t^)Qx(t^)  >  0 


uk 

xT(tk)Qx(tk)  <  0 


tk  £  S 


D, 


a(ty 


>(t) 


(13) 

(14) 

(15) 


where  J\  is  same  as  J  in  (H)  and  J2  =  diag{/,/}. 

By  (O)  -(113),  one  can  see  that  the  event-triggered  switched  control  system  can  be  expressed  as 
a  switched  system  with  impulsive  behaviors  at  switching  instants.  For  the  passive  switching,  that  is 
the  switching  information  is  not  available  and  the  switching  is  supposed  to  possibly  occur  at  every 
switching  sampling  instant,  system  ([E3)-([I3)  can  be  viewed  to  be  under  switching  with  a  dwell  time 
Ts.  The  results  in  pH  E3-EZ3]  for  switched  system  with  dwell  time  can  be  employed.  However,  if  some 
active  switching  is  considered,  which  means  the  switching  rule  is  explicitly  available  to  designed,  the 
passive  switching  result  could  yields  conservativeness,  thus  we  should  improve  these  results  with  the 
aid  of  the  information  of  switching  law.  For  the  active  switching  considered  in  the  remainder  of  paper, 
we  adopt  the  well-known  min-switching  rule,  which  is  described  as  below: 


a(t)  =  argmin  xT  (t)Pix{t) 
ieM 


(16) 


where  Pi  >~  0,  i  £  J\f,  are  matrices  to  be  determined,  see  the  results  in  [ED,  E3].  The  corresponding 
sampled  min-switching  rule  ([ED)  is  described  as 


a(t)  = 


argmin xT (t^)Pix(t^),  tk  £  S 

®{tk)i  t  £  (tkltk- fl) 


(17) 


The  main  aim  of  this  paper  is  to  provide  analysis  and  design  techniques  for  controller,  sampling 
scheme,  and  event-triggering  condition  such  that  the  system  is  stable  with  switching  rule  ([□).  In  the 
following,  the  definition  of  globally  asymptotic  stability  is  presented. 

Definition  1  A  function  7  :  R+  — >  ]R+  is  a  1C  function  if  it  is  strictly  increasing  and  7(0)  =  0,  and 
also  a  function  j3  :  R+  x  R+  — >  R+  is  a  ICC  function  if  for  each  fixed  s  the  function  ft(r,s)  is  a  K, 
function  with  respect  to  r,  and  for  each  fixed  r  the  function  /3(r ,  s)  is  decreasing  with  respect  to  s  and 
/3(r,  s)  — >  0  as  s  — >  0. 

The  definition  of  globally  uniformly  asymptotic  stability  (GUAS)  for  system  (0)-(E3)  is  given 
below. 
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Definition  2  The  equilibrium  x  =  0  of  system  (C3)  (H3)  with  u>(t)  =  0  is  GUAS  under  the  switching 
signal  a(t)  if,  for  initial  condition  x(to),  there  exists  a  class  ICC  function  [3  such  that  the  solution  of 
the  system  satisfies  ||®(t)||  <  /3(||x(fo)||  ,t),  Vf  £  R+. 

In  the  presence  of  input  ui(t),  the  £2-gain  performance  of  system  (O)-(Eil)  is  formulated  in  the 
following. 

Definition  3  For  7  >  0,  system  (£Z3j-(tZ3j  is  said  to  be  GUAS  with  an  C^-gain  performance,  if  the 
following  is  satisfied: 

(1)  System  (tZ3)-(tZ23,)  is  GUAS  when  w(f)  =  0; 

(2)  Under  zero  initial  conditions,  the  following  inequality  holds  for  all  nonzero  to  £  £2(0,00), 

poo  pOO 

/  \\ym2dt<^  /  Mt)\\2dt  (18) 

J to  j to 

where  7  is  called  the  C2~gain. 

Before  ending  this  section,  a  useful  lemma  is  introduced. 

Lemma  1  For  a  matrix  A  £  R"xra  and  a  scalar  Ts  >  0,  there  always  exist  a  sufficiently  large  M*  £ 


N  \  {0},  a  sufficiently  small  e  £  R+  and  matrices  Pm  £  Rnx™,  m  =  {0, ... ,  M},  such  that 

Pm  >-  0,  m  £  {0, . . .  ,M}  (19) 

@1(A,Pm+1,Pm,5)^0,  to  £  {0, . . . ,  M  —  1}  (20) 

9a(A,Pm+1,Pm,5)^0,  to  £  {0, . . . ,  M  —  1}  (21) 

where  S  =  Ts/M ,  hold  for  any  M  >  M* ,  and  Pm,  to  =  {0, . . . ,  M},  have  the  following  form: 

Pm  =  e-ATSmP0e-A5m  -  [  m  e-^T(<5m-t)F(^e-^(<5m-t)(*)di)  m  £  {0, . . . ,  M}  (22) 

J  0 

where  6m  =  mTs/M ,  m  =  {0, . . .  ,M},  and  0  -<  Y(t)  -<  el,  t  £  [0, Ts], 

Proof.  See  Appendix.  □ 


In  this  section,  the  closed-loop  of  event-triggered  switched  linear  system  is  modeled  as  a  switched 
system  with  state  update  at  switching  instant,  along  with  mixed  time-dependent  and  state-dependent 
switching  rules.  In  the  next  section,  the  stability  analysis  will  be  studied  as  the  main  result  in  this 
paper. 


3  Stability  Analysis  for  Event-Triggered  Switched  System 

Motivated  by  the  techniques  used  in  (E3,  EZB,  E3-GUJ  for  switched  systems,  and  [ED|  for  time-delayed  sys¬ 
tems,  the  main  result  for  the  stability  of  event-triggered  switched  control  system  (O)  -(113)  is  presented 
by  the  following  theorem. 

Theorem  1  Consider  event-triggered  switched  control  system  (tZ3)-(H3)  with  io{t)  =  0,  the  following 
three  statements  are  equivalent: 
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(a)  There  exist  scalars  /J/,  >  0,  ft  £  {1, 2},  a  matrix  II  G  and  symmetric  matrices  Pi  >-  0,  i  G  AT, 
such  that 

Hi.h  -<  0,  *GJV,  ftG  {1,2}  (23) 

wj/iere  =  S{AU  Jh,Y!j=\^jiPj,Pi  +  (-l)VhQi) ^),  =  eA^TsQeAiTs . 

(h)  There  exist  scalars  fi/,  >  0,  ft  G  {1,  2},  a  matrix  II  G  and  a  continuous  symmetric  matrix 
function  Pi(t)  :  [0,TS]  — >  R2nx2n)  j  g  J\f,  smc/i  that 


Pift)  y  0,  t  G  [0, Ts\,  i  G)V 

(24) 

3>{Ai,  Pift))  -c  0,  sgA 

(25) 

toi,h  -<  0,  i  G  Af,  he  {1,2} 

(26) 

where  Clith  =  Jj  ^jiPj(0)Jh  -  PZ(TS)  -  (~l)hphQ. 

(c)  There  exist  scalars  Me  N  \  {0},  ph  >  0,  h  G  {1,  2},  a  matrix  n  G  and  symmetric  matrices 

Pi,m  G  R2rax2rl,  to  G  {0, . . . ,  M},  i  e  Af ,  such  that,  for  i  G  Af, 

Pi,m  A  0,  TO  G  {0, ... ,  M} 

(27) 

@l(Ai,  Pi,m+1,  Pi,m,  5)  A  0,  TO  G  {0,  .  .  .  ,  M  —  1} 

(28) 

@2{Ai,Pi,m+l,Pi,m,8)  ^0,  TO  G  {0,  1} 

(29) 

Di,h  A  0,  h  e  {1, 2} 

(30) 

where  S  =  Ts/M  and  flith  =  J J  J2jLi  njiPjfiJh  ~  Pi,M  -  (- l)hphQ ■ 

when  one  of  the  above  equivalent  statements  holds ,  then  system  (IZD,)-(tZD)  with  co(t)  =  0  is  GUAS  with 
switching  signal  m  with  Pi  by  statement  (a),  Pi  =  Pi{ 0)  by  statement  (b)  and  Pi  =  Pi$  by  statement 
(c),  respectively. 


Proof.  The  structure  of  the  proof  is  as  follows:  First,  we  prove  the  equivalence  by  deriving  (c)  => 

(b)  =>  (a)  =>  (c),  then  establish  GUAS  by  (a)  =>  GUAS. 

(c)  =>  (b):  Dividing  interval  I  =  [0,TS]  can  into  M  G  N  \  {0}  segments  described  as  Tm  = 
[8rn .  6m. |_i),  m  =  0, 1, . . . ,  M  —  1,  which  are  of  equal  length  6  =  Ts/M,  and  then  8q  =  0  and  Sm  = 
m 5  =  vj^L-  Based  on  the  discretization  of  I,  the  following  time-scheduled  matrices  Pift),  sgA,  are 
introduced 

j  Pi(t)  =  0-  ~  6(t))Pi,m  P  8(t)Pi,m+l  ,  _  -j  /np 

\  Oft)  =  Mt/Ts  -m  ,tG  m  ’ 

by  which  it  can  be  seen  that  0  <  Off)  <  1  and  Pi  ft)  defines  a  piecewise  linear  matrix  function  over  I. 

By  the  definition  of  Pift),  i  G  Af,  as  (ED),  we  have  P,( 0)  =  P,;.o  and  Pi{Ts)  =  Pi}M ,  so  (E2I)  and  (ED) 
can  make  sure  that  (El)  and  (ED)  hold. 

Then,  one  has 

V+Pi(t)  =  (P.,m+ 1  -  Piim)V+0ft),  t  G  lm  (32) 

Due  to  Oft)  =  Mft  —  Sm)/Ts,  we  have  V+0{t)  =  M/Ts.  Hence  V+Pift)  becomes 

V+P.ft)  =  M(Phm+i  -  Pi,m)/TB,  t  G  lm  (33) 


Thus,  (ESI)  and  (ED)  imply  (ED)  holds. 
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(b)  =>  (a):  Pre-  and  post-multiplying  (ED)  with  e Ai  *  and  its  transpose,  and  integrate  it  over 
[0,  Tg],  it  arrives 

e*?T'Pi(Ta)eA*T‘  -  Pi(0)  -<  0,  ieJV  (34) 

which  implies  P^O)  >-  eA?Ta Pi(Ts)eAiTa ,  i  £  A f.  Furthermore,  it  equals  to 

Pi(Ts)  ~<  e-A*T°Pi(0)e-AiT°,  i  £  Af  (35) 

Using  (ED)  into  (ED),  the  following  inequality  holds  for  i  £  J\T  and  h  £  {1,  2}, 

Jh  Yl"=1  f/i(°)J'*  -  e~AjTaPMe-AiTa  -  (-1  )hnhQ  ■<  0  (36) 

Letting  Pz  =  Pj( 0)  >-  0,  i  £  AT,  (ED)  equals  to 

eAjT‘Jh  Y,"=1  KjipjJheAiT°  -P-  (-1  )hHhQi  -<  0  (37) 

where  Qi  =  eA* Ta  QeAiT‘ .  Thus,  (ED)  can  be  established  by  letting  Pi  =  Pt( 0)  >-  0,  i  £  AT. 

(a)  =3*  (c):  Since  (ED)  holds,  it  implies  that  the  following  inequality  holds 

Jh  Y!}=1  KjiPjJh  -  e-A?T‘Pie-AiT *  -  (-1  )hnhQ  <  0  (38) 

which  implies  that  there  exists  an  e*  >  0  such  that 

Jh  Y,"=1  ^APjJh  -  e-A?T*Pie-AiT°  -  (-1  )hfihQ  -<  —e*  I  (39) 

Then,  for  any  e  >  0,  we  can  let  Pj.o  =  ePi/e*  >-  0,  i  £  AT  (This  choice  of  P^o,  i  £  AT,  maintains  the 
same  switching  law  generated  by  Pi,  i  £  A f.),  and  fa  =  e/z/j/e*  >  0,  h  £  {1,  2},  such  that 

Jn  Y.N]=1  ^iiPjfiJh  -  e-A*T°Pifie-A*T°  -  (-1  )hfaQ  R  -el  (40) 

Using  Lemma  Q],  there  always  exists  a  sufficiently  large  M*  such  that  (E3),  (ED),  (ED)  always  hold  with 
Pi, mi  m  £  {0, . . . ,  M},  M  >  M* ,  i  £  AT,  in  the  form  of 

Pi,m  =  e~A*s™Pi, oe~Ai5™  -  Zi>m,  m  £  {0, . . . ,  M}  (41) 

where 

Zi,m  =  f  m  e~A*  («m-*)y.(t)e-^(«m-0(t)dt 
J  o 

with  6m  =  mTs/M ,  m  =  {0, . . . ,  M},  and  continuous  matrix  functions  T)(t)  >-  0,  i  £  A F. 

Thus,  it  yields 

Pi,M  =  e~A^T“  Pi,oe~AiTs  -  Zi>M  (42) 

Substituting  e~AiTaPi,oe~AiTa  =  Pi,M  +  Zi,M  into  (ED),  we  have 

-r  ■sr^.N  u 

Jh  ^  1  T^jiPjfiJh  —  Pi,M  —  {—  1)  1  faQ  —el  +  Zi,M  (43) 

Since  e  >  0  can  be  arbitrarily  chosen,  we  can  choose  a  sufficiently  large  e  >  0  such  that 

Jh  Y..  x  KjiPjfiJh  —  Pi,M  —  (  —  1)  faQ  <  0  (44) 
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which  implies  that  (BO)  holds. 

(a)  =£*  GUAS:  First,  we  consider  the  system  state  x(tk)  at  sampling  instants,  we  have 
(t+  l  =  i  •Ae4',(t*)Tsz(ifc),  xT  (tk+1)Qx{tk+1)  >  0 

\Lk+ 1)  \  A  4-  T,  -r- 

he  x(t+),  xT (tk+1)Qx{tk+1)  <  0 


(45) 


x(k  +  1)  = 


(46) 


Due  to  x{tk+l)  =  e  a(T )  “  x(tk),  and  letting  Qi  =  eAi  TsQeAiTs ,  k  =  tk,  x(k)  evolves  according  to  the 
following  dynamics 

JleA<’<-k^Tsx(k),  xT (k)Qix(k)  >  0 
j2eA”(k'>Tax(k),  xT  (k)Qix(k)  <  0 

where  er(fc)  =  argmin xT  (k)Pii(k)  and  Pi;  i  G  AT,  is  same  as  in  switching  signal  (QIZI). 

Construct  Lyapunov  function  candidate  as  V(x(k))  =  xT  {k)Pa(k)x{k)  and  define  A V(x(k))  = 
V{x{k  +  1))  —  V(x(k)),  under  the  min-switching  law  (DZZI),  we  have 

A  V(x(k))  =  minxT(fc  +  T)Pjx(k  +  1)  —  xT  (k)Pix(k) 
je  M 


<  xT (k  +  1)  ±  njiPjSJ  +  1)  —  xT (k)Pi 


x(k) 


By  (SO),  A V(x(k))  arrives 


NV(T(kA  =  I  iT(fc)rd xT(k)QjX{k)  >  0 
'  xT  (k)Ti}2x(k),  xT (k)Qi,x(k)  <  0 


(47) 


where  rf)i  =  S{A^  Ji,  J2j=i  pi,  Ts),  rij2  =  J2,  J2j=i  njipj,o,  Pi,  Ts).  Since  (E3)  holds,  it 

implies  there  exists  a  sufficiently  small  e  >  0  such  that  S i}h  <  —el,  Vi  G  J\f,  h  G  {1,2}  then  using 
S-Procedure,  it  ensures  that 

A V{x{k))  <  -e  ||i(fc)H2  ,  k  G  N  (48) 

Letting  Am;n,  Amax  be  the  minimal  and  maximal  eigenvalues  of  Pi,  i  G  A f,  respectively,  it  implies  that 
Amin  ||i(&)||2  <  V(x{k))  <  Amax  ||i(fc)||2.  Thus,  (53)  implies  that  V{x{k))  <  (1  -  e/\max)kV(x(t0)), 
where  0  <  1  —  e/Amax  <  1.  Due  to  k  =  tk/Ts,  one  has 

o(tk-t0)\n  WW. 


V{x(t+))  <  el 


V(x(t0)),  tk  eS 


Furthermore,  it  arrives 


'Mill  <  l/P''*'''’  PKWII,  1.E5 

V  ^min 


(49) 

(50) 


where  p  =  -  ln(l  -  e/Amax)/2 Ts  >  0. 

Then,  let  us  consider  any  t  G  {tk,tk+ 1),  the  dynamics  of  mode  i  yields  x(t)  =  eAi<d~tk>  x{tk) , 
t  G  (tk,tk+ 1).  Using  the  following  derivation 


0Ai(t-tk) 


<  e||A(t-tfe)||  <  e||A||T%  t&  (tfcjtfc+1) 


(51) 


we  have  ||x(i)||  <  c||x(tfc)||,  t  G  {tk,tk+ 1),  where  c  =  maxjewy  ell^iHTs.  Thus,  by  (EO),  it  can  be  ob¬ 
tained  that  ||x(i)||  <  Ce~p^t~t°'>  ||x(fo)||,  where  C  =  cepTa  \J Amax/Amin  >  0,  and  the  GUAS  can  be 
established  by  the  existence  of  ICC  function  /3(||x(t0)||  ,t )  =  Ce~p('t~t°')  ||x(f0)||-  □ 


Some  observations  are  obtained  for  three  conditions  in  Theorem  ED: 
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1.  If  no  event-triggering  condition  is  considered  and  the  state  x(t)  updates  at  each  sampling  instant, 
event-triggered  system  (0)-([E3)  is  reduced  to  (EI)-(O),  and  as  a  result,  (ED)  can  be  rewritten  to 

^ - ■yN 

S’ (A,  J,  E  Aiphph  Ta)  <  0,  i  G  A  (52) 

It  can  be  found  that  (E2)  recovers  the  result  in  [E3] ,  which  deals  with  the  switched  system  with 
min-switching  law  ([EO)  only  acts  at  sampling  instant  tk ■  Theorem  ffl  generalizes  the  sampled 
switching  case  to  event-triggered  switching  case.  Furthermore,  if  we  consider  the  passive  switch¬ 
ing,  which  means  switched  system  could  switch  to  any  subsystems  at  every  switching  instant  tk- 
That  means,  for  any  j  ^  i ,  i,j  G  A f,  we  have  to  let  nji  =  1  and  7 rpi  =  0,  p  ^  j,  so 

S(Ai,  J,  Pj,  Pi,Ts)  ~<  0,  i,j£Af  (53) 

which  exactly  recovers  the  result  in  [ED]. 

The  basic  idea  of  Condition  (a)  is  to  consider  the  evolution  of  system  state  at  sampling  instant 
tk,  and  the  asymptotic  convergence  of  x{tk)  guarantees  the  asymptotic  stability  of  system  (03)- 
(\m).  However,  if  one  attempts  to  make  some  further  extensions  of  Condition  (a)  such  as  robust 
sampling  case  and  £2-gain  performance  analysis,  the  presence  of  exponential  term  eAiTs  makes 
such  extensions  difficult. 

2.  Condition  (b)  basically  is  an  extension  of  the  result  in  [ES],  from  dwell  time  switching  to  peri¬ 
odically  event-triggered  switching.  Regardless  of  event-triggering  condition,  system  (□)-(□)  is  a 
switched  system  with  a  periodic  dwell  time  Ts,  and  if  we  deactivate  the  switching  rule  (O)  to 
consider  passive  switching,  it  leads  to  n,ji  =  1  and  npi  =  0,  p  ^  j,  thus  (ED)  is  rewritten  to 

p0{ o)  -  P{TS)  -<  0,  j  ±  i,  i,j  G  A f  (54) 

Together  with  (ESI),  (ESI),  the  result  in  m  is  recovered. 

Still  consider  system  (□)-  (0)  regardless  of  event-triggering  condition,  (ESI)  becomes 

,-^iV 

J2j=1^PM-P(Ts)^0  (55) 

Then,  let  us  consider  the  special  case  with  sampling  interval  Ts  — >  0.  In  this  case,  we  have  to  let 
the  continuous  matrix  function  Pi(t)  =  Pi,  i  G  AT,  then  (ESI)  implies  @{A,Pi)  =  eS'(Ai,Pi),  and 
(ED)  arrives  at 

Ei=17 TiiPj-Pi^O  (56) 

From  the  fact  of  njiPj  ~  Pi  =  </> jiPj ,  $  G  M^,  i,  j  G  AT,  (ED)  leads  to 

E  AnPj*  0  (57) 

z — '7  =  1 

Combining  (ED),  (E3),  the  following  result  can  be  established 

, — ,N 

V(Ai,  Pi)  +  E  -=1  <f>ji Pj  -<  °,  $  G  Mf ,  i,j  G  Af  (58) 

which  exactly  recovers  result  in  [ED]  for  min-switching  rule.  Therefore,  Condition  (b)  is  an 
extension  to  sampling  case  and  further  to  event-triggered  case.  One  point  need  to  be  noted  this 
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Table  1:  Computational  complexities  of  Conditions  (a),  (b)  and  (c)  with  fixed  II  £ 


Number  of  Variables 

LMI  Constraints 

Condition  (a) 

(4n2N  +  2nN)/2  +  2 

6  nN  +  2 

Condition  (b) 

OO 

OO 

Condition  (c) 

(4n2  +  2n)(M  +  l)/2  +  2 

6  nN(M  +  1)  +  2 

min-switching  law  may  introduce  Zeno  behaviors,  but  if  we  let  Ts  be  a  positive  constant  in  our 
periodic  event-triggered  rule,  one  advantage  is  the  elimination  of  Zeno  behavior  in  switching. 

In  comparison  with  Condition  (a),  Condition  (b)  does  not  have  any  exponential  terms  which 
facilitates  its  further  extensions  to  solve  other  problems.  However,  it  is  not  numerically  testable 
to  check  the  existence  of  such  time- varying  matrix  functions  Pl(t).l  i  £  A f . 

3.  Condition  (c)  is  a  discretized  version  Condition  (b),  and  similar  as  what  has  been  discuss  for 
Condition  (b),  if  we  discard  the  event-triggering  condition  and  (ED)  becomes 

T  1  KjiPjfi  -  Pi,M  -<  o  (59) 

which  recovers  the  result  in  [E2I].  Moreover,  if  we  further  deactivate  the  min-switching  strategy, 
(ED)  can  be  reduced  to 

Pj, o  —  P%,m  <  0  (60) 

to  recover  the  result  in  m  for  switched  system  under  dwell  time  constraint. 

With  a  particularly  constructed  Pi(t),  i  £  Af,  Condition  (c)  recasts  the  search  for  a  continuous 
matrix  function  Piit)  as  a  finite  number  of  matrices  m  £  {0,  i  £  Af,  which  is 

solvable  for  many  current  tools. 


4.  Though  the  three  conditions  are  equivalent,  the  computation  complexities  are  different.  Condi¬ 
tion  (a)  looks  simpler  and  computationally  much  more  efficient,  see  Table  1  for  the  comparison  of 
computational  complexities  with  a  prescribed  n  £  M^.  Condition  (b)  is  actually  not  numerically 
tractable  by  the  present  tools,  so  a  special  structure  of  Piit),  i  £  AT,  is  employed  in  Condition 
(c),  it  turns  the  infinite  number  of  decision  variables  in  time- varying  Pj(t),  i  £  A f  into  a  finite 
number  of  matrices  pjm,  m  £  {0, . . . ,  M},  i  £  Af.  However,  the  equivalency  of  Condition  (c)  to 
Conditions  (a)  and  (b)  has  to  be  established  based  on  a  sufficient  large  M,  and  the  computation 
cost  increases  as  M  grows,  see  Table  1.  Though  more  computation  cost  has  to  pay  in  Condition 
(c),  the  further  extensions  beyond  stability  become  possible. 


Example  1  Consider  a  switched  system  with  two  modes 


"  Ai  ' 

’  1 

6 

3 

-2 

A2 

5 

.  B 2  . 

1 

0.5 

-1.3 

-1.6 

-3.3 

0.3 

0.2 

0.3 

The  feedback  gains  are  K\  =  [—5.1744  —  5.1904]  and  K2  =  [18.7593  16.3442],  which  ensure 

the  Ai  +  BiKi,  i  £  {1,2},  are  Hurwitz  stable.  The  event  triggering  condition  is  T(x(tk),x(tk))  = 
\\x(tk)  —  x{tu)\\  —  A||a;(£fc)||,  where  A  >  0.  To  search  for  n  £  M^,  we  define  7 rn  £  [0,1]  and 
7Ti2  €  [0,1],  then  7T2i  =  1  —  7Th  and  tt 22  =  1  —  7Ti2,  respectively.  The  increments  dnu  =  0.1  and 
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M=17 


Figure  2: 


The  least  values  of  sufficiently  large  parameter  M  for  Condition  (c)  to  verify  GUAS 


Table  2:  Computational  time  (second)  of  Condition  (c)  with  a  fixed  II  G 


Ts  =  0.1 

Ts  =  0.2 

CO 

O 

II 

o 

II 

Ts  =  0.5 

A  =  0.1 

3.045 

4.842 

6.235 

7.682 

12.372 

A  =  0.2 

3.767 

4.881 

6.349 

8.628 

13.680 

A  =  0.3 

3.624 

6.349 

8.932 

9.158 

14.046 

A  =  0.4 

3.814 

5.817 

9.434 

16.745 

16.750 

A  =  0.5 

3.983 

9.738 

12.186 

20.909 

29.081 

dir  12  =  0.1  are  taken  to  divide  [0, 1],  and  use  the  discretized  points  to  turn  the  conditions  in  Conditions 
(a)  and  (c)  into  LMI  feasibility  problems. 

First,  we  use  Condition  (a)  to  verify  that  the  GUAS  can  be  established  with  sampling  times  Ts  = 
{0.1,  0.2,  0.3,  0.4,  0.5}  and  state  error  A  =  {0.1,  0.2,  0.3,  0.4,  0.5}.  Then,  to  show  the  equivalence, 
we  use  Condition  (c)  to  obtain  same  GUAS  results,  provided  with  sufficiently  large  parameters  M .  The 
results  are  shown  in  Figure  2. 

Figure  2  shows  the  existence  of  sufficiently  large  M  ensuring  the  equivalence  of  Conditions  (a) 
and  (c).  However,  the  computational  complexities  of  two  theorems  are  different.  The  computational 
complexity  of  Condition  (a)  is  fixed  if  the  number  of  subsystems  and  system  order  are  fixed,  as  Table 
1  shows,  but  the  computational  complexity  of  Condition  (c)  increases  as  M  grows.  The  computational 
time  is  given  in  Table  2.  Larger  A  or  Ts  will  lead  to  more  computational  time  which  is  listed  in  Table  2 
is  because  larger  A  or  Ts  needs  larger  M  to  establish  the  stability,  as  what  Figure  1  shows.  Taking  the 
Ts  =  0.2  for  example ,  A  =  0.2  needs  M  =  2  and,  on  the  other  hand,  A  =  0.3  needs  M  =  4.  Larger  M 
has  more  computational  complexities  as  shown  in  Table  1.  If  the  M  are  same,  e.g.  the  case  Ts  =  0.1, 
A  =  0.2  and  A  =  0.3  both  need  M  =  2,  so  the  computational  times  are  similar. 
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Despite  the  equivalence  of  Conditions  (a),  (b)  and  (c),  the  main  advantage  of  Condition  (c)  lies 
in  its  convenience  of  extending  to  solve  further  problems.  In  next  sections,  extensions  will  be  made 
to  robust  sampling  case  and  £2-gain  performance  analysis  for  event-triggered  switched  control  system 
based  on  Condition  (c). 


4  Robust  Sampling  Scheme 


In  this  section,  the  uncertainties  in  sampling  interval  will  be  considered.  To  further  develop  a  robust 
switching  rule  (DZZI),  the  sampling  interval  is  generalized  to  Ts  G  [Trnin ,  Tmax] .  Similar  as  the  general¬ 
ization  from  periodic  switching  to  aperiodic  switching  in  [BO] ,  the  generalization  of  Conditions  (a)  and 
(b)  in  Theorem  ffl  can  be  made  simply  by  replace  a  fixed  Ts  by  a  variable  r  €  [Tmin,Tmax]  in  these 
conditions.  For  instance,  Condition  (a)  can  be  directly  generalized  as 

Jh,  Y  i  -KjjP^Pj  +  (-1  )h/jheA*TQeAiT,  r)  -<  0,  i  G  Af,  he  {1,2}  (61) 

holds  for  all  r  G  [Tmin,  Tmax].  However,  it  is  difficult  to  check  (EH)  for  all  r  G  [Tmin ,  Tmax]  which 
has  infinitely  many  number  for  checking  in  an  interval  [Tmjn,Tmax],  due  to  the  continuity  argument 
and  intricate  dependence  of  (EH)  with  r  G  [Tmin,  Tmax].  Thus,  it  is  difficult  to  numerically  verify  the 
stability  by  (EH)  which  actually  requires  infinite  values  for  checking. 

In  order  to  establish  a  numerically  tractable  method  for  robust  sampling  interval  Ts  G  [Tmin,  Tmax], 
we  resort  to  generalize  Condition  (c).  Like  the  extension  from  dwell  time  to  ranged  dwell  time  in  [ED] 
for  sampled-data  systems,  the  following  theorem  can  be  developed  for  robust  sampling  interval  in  the 


framework  of  event-triggered  control  scheme. 

Theorem  2  Consider  event-triggered  switched  control  system  (tZ3)-(tZ3)  with  uj(t)  =  0,  if  there  exist 
scalars  Me  N\  {0},  ft  >  0,  /i  G  {1, 2},  a  matrix  n  G  and  symmetric  matrices  Pi^m  G  JR2™*2™, 
me  {0, . . . ,  M},  i  e  A f,  such  that,  for  i  G  A f, 

Pi,m  >-  0,  m  G  {0, . . . ,  M}  (62) 

S>i{Ai,Pitm+1,Pitm,6)  -<  0,  m  e  {0, . . .  ,M  -  1}  (63) 

@2(Ai,Pi,m+uPi,m,8)  ~<  0,  to  G  {0, 1}  (64) 

ttith,rh  A  o,  TO  G  {M,  ■  •  ■ ,  M},  h  e  {1,  2}  (65) 


where  6  =  Tmax/AT,  M  =  int{  }  and  =  Jh  SyLi  njiPj,oJh-Pi,rh,-(-^)hl^hQ,  then  system 

(El)- (H3J  with  co(t)  =  0  is  GUAS  under  sampled  switching  rule  if TH)  with  Pi  =  Pi,o,  i  G  Af. 

Proof.  Since  M  =  int{  AjT™in},  we  have  — <  Tm in  which  implies  that  the  interval  [Tmin,  Tmax]  C 
Um=M,...,M-l  Pm- 

Considering  Pi(t),  t  G  [0,Tmax]  defined  by 


r  PS)  =  (i  -  o{t))pitm  +  e(t)phm+ 1 
\  6(t)  =  Mt/Tmax  -  m 


where  0  <  9(f)  <  1.  First  by  (E2I),  we  can  obtain  Pi(t)  >-  0,  t  G  [Tmjn,Tmax].  Then,  (EH)  and  (E3)  have 
T>(Ai,Pi(t ))  -<  0,  and  for  any  r  G  [Tmjn,Tmax],  it  is  obtained 

P,o  =  Pi(0)  >-  e^TPi(r)eAiT,  r  G  [Tmin,  Tmax]  (67) 
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(68) 


by  integrating  T>{Ai,  Pj(t))  -<  0  over  [0,  r].  Then,  (ESI)  implies  that 


Pj,oJhZAi  T  -  eAi  T Pi(r)eAi  T  -  (-1  )VQ(t)  A  0 


A.1  T 


J=1 


holds  for  r  £  [Tmjn, Tmax],  where  A(r)  =  eAi  TQeAiT.  Using  (E3)  into  (ESI)  and  letting  P,;  =  Pji0, 
i  £  A/”,  it  reaches  that 


<?(A,  Jh,^2j=1^jiPj,Pi  +  (-^)hl^hQ(r),T)  -<  0,  r  £  [Tmin,  Tmax] 


which  is  exactly  (OH) ,  thus  the  robust  GUAS  can  be  established. 


(69) 

□ 


In  comparison  with  (ED,  the  extension  of  Condition  (a),  which  has  an  infinite  many  decision 
variables  to  search,  Theorem  □  only  has  a  finite  number  of  decision  variable  to  check  the  GUAS  for 
event-triggered  switched  system  with  ranged  sampling  intervals.  The  numerically  tractable  feature 
is  an  obvious  advantage  over  (EU)  which  is  a  straightforward  extension  from  Condition  (a),  and  this 
promising  feature  of  Theorem  □  which  is  actually  a  generalization  of  Condition  (c)  basically  benefits 
from  the  fact  that  the  system  matrices  Ai  are  affine  in  the  corresponding  conditions. 


5  /^2-Gain  Performance  Analysis 

In  the  presence  of  disturbance  co(t),  A-gahr  performance  is  a  disturbance  attenuation  performance 
for  event-triggered  switched  system  (0)-([E3).  The  basic  idea  of  Condition  (a)  in  Theorem  □,  that  is 
abstracting  continuous-time  system  (0)-([E3)  into  a  discrete-time  version,  is  difficult  to  be  extended 
from  stability  analysis  to  A-gain  performance  analysis,  since  the  discrete-time  abstraction  only  de¬ 
fines  the  input-output  relation  at  sampling  instants  tk,  losing  the  information  over  interval  (tk,tk- t-i)- 
Moreover,  the  technical  difficulties  for  extension  mainly  lies  in  the  exponential  term  eAiTa .  On  the 
other  hand,  Condition  (c)  in  Theorem  ffl  can  be  extended  owing  to  the  affineness  in  system  matrix  Ai. 
In  the  following,  a  numerically  tractable  result  is  proposed  for  A-gahi  performance  analysis. 

Theorem  3  Consider  event-triggered  switched  control  system  (tZ3)  (G3),  if  there  exist  scalars  M  £ 
N\{0},  Hh  >  0,  h  £  {1,  2},  a  matrixH  £  and  symmetric  matrices  Pi^m  £  ]R2rax2rl;  m  £  {0, . . . ,  AI}, 
i  £  A f,  such  that,  for  i  £  J\f , 

Pi,m  >-  0,  m  £  {0, . . 

^i,m, l  A  0,  m  £  {0, . . 

^i,m, 2  A  0,  TO  £  {0,  .  . 

A  .ft  ^0,  ft  £  {1,  2} 

where  —  J ^  )  b — i  TCaPraJh  Pi,M  (  1)  t^hQ,  and 


'  9>l(Ai,  Pi^m+l,  Pi,m,  Ts/M) 

* 

* 

Ej  Pi,m+ 1 

“T  2I 

* 

Ci 

A 

-I  _ 

'  @2  {A,,  Pi, rn+l,  Pi, m,Ts/M) 

* 

* 

Ej  Pi,m 

-7  2I 

* 

Ct 

A 

-I  _ 

14 

,  M} 

(70) 

,M  —  1} 

(71) 

,M-  1} 

(72) 

(73) 
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then  switched  system  (G3)  (tZ H)  is  GUAS  and  has  an  C^-gain  7  under  sampled  switching  rule  m)  with 
Pi  =  Pi, a,  i  G  AT. 

Proof.  The  GUAS  can  be  easily  obtained  by  Condition  (c)  in  Theorem  □,  thus  we  focus  on  the 
£2-gain  performance  in  the  following.  First,  we  let 

n(*)  =  l|y(t)ll2-72IM*)||2  (74) 

and 

Jk(t)=  f  fi(s)ds,  t  G  [tfe,tfe+1]  (75) 

which  can  imply  that 

=  [  (Ct(s) +V+Vi(x(s)))  ds-Vi(x(t~)) +  Vi(x(t^))  (76) 

Jtt 

where  Vi(x(t))  is  defined  as  Vi(x(t))  =  xT (t)Pi(t)x(t),  i  G  Af,  with  Pi(t),  i  €  Af,  defined  by  (ED). 
Then,  by  (113),  it  can  be  deduced  that  U(s)ds  =  Jfc(t^+1),  which  can  be  rewritten  as 


f  tt(s)ds  =  J2k=o  f  (l n{s)  +  V+Vi(x(s)))ds  +  J2k=i(Vj(x(t^))-Vi{x(tk)))+Vi(x(t0 )) 

J  to  Jtk 

(77) 

From  (EDI),  one  has 

Vj(x(t+))  -  Vi(x(t^))  <  0,Vtfc  G  5  (78) 


is  satisfied  with  min-switching  rule  (ED).  Moreover,  it  is  obtained  that 


n(t)  +  v+Vi(x(t))  =  cT{t) 


A»  Pi{t)Ei  +  CTDi 

*  DjDi-j2! 


(79) 


where  CT  =  [xT (t)  wT(f)],  A,  =  S>{Ai,Pi{t))  +  Cj Ci. 

Thus,  from  (ED),  (E2),  it  gives  Et{t)  +  V+Vi(x(t))  <  0.  Together  with  (ESI)  and  x (t0)  =  0,  we  obtain 


fl(s)ds  <  0 


(80) 


which  leads  to  ||y(f)||2df  <  y2  J)°°  ||w(f)||2dt  when  cu(t)  ^  0.  Therefore,  the  £2-gain  performance 
is  guaranteed.  The  proof  is  complete.  □ 


From  Theorem  3,  it  should  be  stressed  that  although  the  min-switching  rule  (ED)  only  acts  at 
sampling  instants  tk  G  S ,  the  £2-gain  level  which  is  defined  over  [to,  00)  can  be  estimated.  This 
is  because  (ED)  and  (E2)  fully  characterize  the  input-output  property  in  the  sense  of  £2_gain  during 
[tk,tk+ 1).  Moreover,  if  the  robust  sampling  scheme  is  considered,  the  similar  extension  can  be  easily 
made  as  Theorem  □. 

Under  the  framework  of  Theorem  0,  an  estimate  of  the  £2-gain  can  be  obtained  by 

min  7  2 

(81) 

s.t.  (ED),  (ED),  (E3),  (EDI) 
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Figure  3:  Suboptimal  /Vgain  7  with  respect  to  different  M 


Same  as  the  stability  analysis  result,  the  computational  results  obtained  by  solving  the  linear-matrix- 
inequality-based  optimization  problems  (EH)  also  depend  on  the  choice  of  M.  Less  conservative  results 
will  be  obtained  with  larger  M ,  at  the  expense  of  higher  computational  cost,  which  will  be  shown  by 
the  following  example. 

Example  2  Consider  a  switched  system  with  two  modes  same  as  in  Example  1,  and  Ci,  Di,  Ei, 
i  £  {1,  2}  are  chosen  as  below: 


C1=C2  =  [1  1],  E1  =  E2 


0.2 

0.5 


D\  —  D2  —  0.5 


(82) 


We  still  consider  7Tn  £  [0, 1]  and  tt\2  £  [0, 1]  with  tt2i  =  1  —  7 rn  and  n22  =  1  —  7172,  respectively, 
the  increments  A7Tn  =  0.1  and  A7r12  =  0.1  are  taken  to  divide  [0,1],  and  search  the  optimal  7  for 
these  discretized  points  by  Theorem  □.  The  suboptimal  C2-gain  is  obtained  as  the  minimal  value  of 
the  optimal  7  of  all  discretized  points.  Furthermore,  given  a  constant  sampling  time  Ts  =  100  ms  and 
A  =  {0.1,  0.2,  0.3,  0.4,  0.5},  the  suboptimal  C2-gain  7  with  respect  to  different  M  are  shown  in  Figure 
3.  From  Figure  3,  it  can  be  observed  that  the  estimated  C2-gain  7  decreases  as  M  increases,  this  is 
because  that  a  larger  M  implies  a  finer  division  of  the  sampling  interval,  and  thus  a  less  conservative 
result  can  be  obtained.  Moreover,  it  can  be  also  found  that  the  control  performance  becomes  worse 
with  a  larger  state  error  A  in  event  trigger  condition,  this  is  consistent  with  the  actual  situation.  The 
increasing  computational  complexities  along  with  M  is  same  as  in  Table  2,  which  is  not  presented  here. 


6  Conclusions 

In  this  paper,  the  event-triggered  control  for  switched  linear  systems  has  been  studied.  Three  stability 
criteria  are  proposed  to  ensure  asymptotic  stability  of  switched  system  subject  to  min-switching  rule 
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which  is  only  allowed  to  activate  at  sampling  instants.  It  has  been  proved  that  the  three  stability 
criteria  are  equivalent.  Then,  taking  advantages  of  one  stability  criterion  with  affineness  in  system 
matrices,  extensions  to  robust  sampling  scheme  and  £2-gahr  analysis.  In  the  future  work,  the  controller 
design,  switching  rule  design  and  event-triggering  condition  design  should  be  taken  into  account  based 
on  the  stability  analysis  results  proposed  in  this  paper. 
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A  Proof  of  Lemma  CD 

First,  given  (E3)  and  0  -<  Y(t)  -S  e/,  t  £  [0,  Ts],  with  a  sufficiently  small  e  >  0,  obviously  we  can  obtain 
Pm  >-  e-AT6mP0e~ASm  -  e  f  ™  e~AT y  Q)  m  g  {0, . . . ,  M} 

Jo 

holds  for  any  initial  Pq  >-  0. 

Letting  Zm  =  e~AT (t)e~A('Sm~t^t^ dt  and  substituting  (E2)  into  @i(A,  Pm+i,Pm,6)  to 
get 

@i(A,Pm+1,Pm,6)  =  #m,1(6)+#m,2(S)  +  #m,3(6)  (83) 

where  5  =  TS  /M  and 

VmA5)=e-AT5™n(h)e~AS™ 

Tl(5)  =  IS  (A,  P0)  +  S' (A ,  I ,  Pq/S,  Pq/S,  S ) 
l9m,2(S)  =  -tS(A,Zm ) 

'dm,  3(8)  =  {Zm  —  Zm+i)/S 

Due  to  lim,5_>o+  sup S(A,  I,  P0/5,  Pq/6,5)  =  — ^(A,P0),  therefore  it  yields  that  lim(5_>o+  supO(<5)  =  0, 
which  implies 

lim  sup$m  1(6)  =0  (84) 

<S-K)+ 

Moreover,  due  to  0  <  Srn  <  Ts,  it  implies  that  e~ASm  is  bounded,  dm,i(S)  uniformly  converges  to  zero. 
In  addition,  it  can  be  seen  that 


lim  supi?m>3(6)  =  -Y(Sm)  +  cS{A,Zm) 

6— >•  0+ 

(85) 

which  results  in 

lim  sup(^mi2(<5)  +  i?m,3((5))  =  —Y(Sm) 

5^-0+ 

(86) 

which  implies  that  lim^_>0+ 

sup (&i,m,2(6)  +  dm, 3(d))  A  0  due  to  Y(t)  >-  0,  t  €  [0,TS], 
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In  conclusion,  with  the  aid  of  (EH)  and  (ED),  we  have 


lim  sup@i{A,Pm+1,Pm,5)  -<  0 

<5 — >0+ 


(87) 


so  there  exists  a  sufficiently  small  S*  such  that 


&l(A,  Pm+l,  Pmi  $)  A  0 


(88) 


holds  for  all  6  <  6*. 

By  a  similar  procedure  as  above,  we  can  consider  ^(A,  Pm+1,  Pm,  8)  to  obtain 


lim  sup@2(A,Pm+1,Pm,8)  -<  0 

S— >o+ 


(89) 


and  we  can  find  a  sufficiently  small  8%  such  that 


9i{A,Pm+1,Pm,8)  -<  0 


(90) 


holds  for  all  <5  < 

By  setting  S*  =  min{JJ,  <5J},  we  can  conclude  that  there  exists  a  sufficiently  small  6*  such  that  (ED) 
and  (EH)  hold  for  any  S  <  8*.  Due  to  5  =  Ts/M,  it  is  equivalent  to  the  existence  of  a  sufficiently  large 
M*  such  that  (ED)  and  (EH)  hold  for  any  M  >  M*. 
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Output  Reachable  Set  Estimation  for  Switched 
Linear  Systems  and  Its  Application  in  Safety 

Verification 

Weiming  Xiang,  Hoang-Dung  Tran,  and  Taylor  T.  Johnson 


Abstract — This  paper  addresses  the  output  reachable  set  es¬ 
timation  problem  for  continuous-time  switched  linear  systems 
consisting  of  Hurwtiz  stable  subsystems.  Based  on  a  common 
Lyapunov  function  approach,  the  output  reachable  set  is  estimat¬ 
ed  by  a  union  of  bounding  ellipsoids.  Then,  multiple  Lyapunov 
functions  with  time-scheduled  structure  are  employed  to  estimate 
the  output  reachable  set  for  switched  systems  under  dwell  time 
constraint.  Furthermore,  the  safety  verification  problem  of  uncer¬ 
tain  switched  systems  is  investigated  based  on  the  result  of  output 
reachable  set  estimation.  First,  a  sufficient  condition  ensuring  the 
existence  of  an  approximate  bisimulation  relation  between  two 
switched  linear  systems  with  a  prescribed  precision  is  proposed. 
Then,  the  safety  verification  for  an  uncertain  switched  system 
can  be  performed  through  an  alternative  safety  verification  for  a 
switched  system  with  exact  parameters.  Numerical  examples  are 
provided  to  illustrate  our  results. 

Index  Terms — Reachable  set  estimation,  safety  verification, 
switched  system,  uncertain  system. 


I.  Introduction 

Switched  systems  are  a  typical  class  of  hybrid  system- 
s,  which  consist  of  a  family  of  subsystems  described  by 
continuous  or  discrete-time  dynamics,  and  a  switching  law 
that  specifies  the  active  subsystem  at  each  time  instant.  Due 
to  the  multi-modal  feature,  switched  systems  can  efficiently 
model  practical  systems  that  are  inherently  multi-modal,  i.e., 
several  dynamical  subsystem  models  are  required  to  describe 
their  behaviors.  So  far,  the  research  on  switched  systems  has 
attracted  significant  attention  and  an  extensive  literature  is  by 
now  available,  for  example  in  stability  and  stabilization  [1]- 
[5],  controllability  and  reachability  analysis  [6],  TLoo  control 
and  filtering  [7]— [9]. 

Reachable  set  estimation  aims  to  derive  a  closed  bounded 
set  that  constrains  all  the  state  trajectories  generated  by  a 
dynamic  system  with  a  prescribed  initial  state  set  and  an 
input  set.  As  its  further  extension,  the  output  reachable  set 
estimation  is  to  derive  a  closed  bounded  set  containing  the  set 
of  all  outputs  of  a  system.  Reachable  set  estimation  problem 
is  not  only  of  theoretical  interest  in  robust  control  theory 
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[10],  but  also  closely  related  to  practical  engineering  for  the 
safety  verification  problems  [11],  In  some  early  work,  the 
reachable  set  bounding  was  considered  in  the  context  of  state 
estimation  and  it  has  later  received  a  lot  of  attention  in  pa¬ 
rameter  estimation,  see  [12]  and  references  therein.  Recently, 
many  researchers  have  been  interested  in  employing  ellipsoidal 
techniques  based  on  Lyapunov  function  approaches  to  estimate 
the  reachable  sets  for  different  classes  of  systems.  In  the 
framework  of  bounding  ellipsoid,  the  quadratic  Lyapunov 
function  has  played  a  fundamental  role  in  the  reachable  set 
estimation  problem,  and  it  has  been  further  developed  to  time- 
delay  systems  [13]— [16],  singular  systems  [17],  discrete-time 
switched  systems  under  arbitrary  switching  [18]  and  periodic 
switching  [19].  However,  according  to  the  best  of  the  authors’ 
knowledge,  the  reachable  set  estimation  for  continuous-time 
switched  systems  with  dwell-time  restriction  has  not  been  fully 
investigated,  and  it  therefore  motivates  our  study. 

In  this  paper,  the  contributions  are  two  folds.  First,  we  study 
the  output  reachable  set  estimation  problem  for  continuous¬ 
time  switched  linear  systems  consisting  of  Hurwitz  stable  sub¬ 
systems.  In  the  arbitrary  switching  case,  an  over  approximation 
of  output  reachable  set  is  obtained  as  a  union  of  a  collection 
of  bounding  ellipsoids  centered  around  origin  and  moreover, 
a  linear  matrix  inequality  (LMI)  based  optimization  problem 
is  formulated  to  obtain  the  smallest  estimated  reachable  set. 
These  results  are  all  derived  in  the  framework  of  a  common 
Lyapunov  function  shared  across  modes,  however,  it  may  yield 
overly  conservative  results,  especially  when  some  information 
of  switching  laws  is  available.  Thus,  with  regard  to  a  class  of 
time-dependent  switching  signal  under  dwell  time  constraint,  a 
time-scheduled  multiple  Lyapunov  function  approach  is  further 
employed  and  preciser  estimation  results  can  be  achieved. 
In  particular,  it  is  worth  mentioning  that  this  time-scheduled 
multiple  Lyapunov  function  approach  covers  the  common  Lya¬ 
punov  function  approach.  In  some  papers,  e.g.,  [20],  [21],  the 
finite-time  boundedness  is  used  for  bounding  state  trajectories 
of  a  system,  but  it  focuses  on  a  finite-time  interval  other 
than  all  time  along  the  system  operation.  Furthermore,  the 
estimation  from  initial  time  to  infinity  is  necessary  for  some 
problems  such  as  the  bisimulation  and  safety  verification  in 
the  second  contribution  in  this  paper. 

Based  on  the  results  for  output  reachable  set  estimation 
and  inspired  by  approximate  bisimulation  relations  in  [22]— 
[24],  a  sufficient  condition  is  derived  to  establish  the  existence 
of  approximate  bisimulation  of  two  switched  linear  systems. 
Then,  since  the  safety  verification  for  uncertain  systems  is 
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difficult  due  to  the  uncertain  time-varying  coefficients  in  the 
system  matrices,  one  would  ask:  Can  we  find  a  bisimilar 
system  with  exact  parameters  for  an  uncertain  system  and 
perform  a  safety  verification  for  the  bisimilar  system  to  ensure 
the  safety  of  the  uncertain  system?  In  this  paper,  an  LMI-based 
method  is  proposed  to  convert  the  uncertain  switched  system 
into  a  switched  system  with  exact  parameters  along  with  a 
precision  between  two  systems,  so  that  the  safety  verification 
for  uncertain  systems  can  be  performed  by  verifying  the 
safety  of  the  transformed  systems,  avoiding  the  difficulties  in 
handling  the  uncertainties. 

The  rest  of  this  paper  is  organized  as  follows.  Some  prelim¬ 
inaries  and  problem  formulation  are  given  in  Section  II.  The 
main  results  on  output  reachable  set  estimation  is  proposed  in 
Section  III.  In  Section  IV,  the  application  to  safety  verification 
for  uncertain  switched  systems  is  presented.  Conclusions  are 
given  in  Section  V. 

Notation:  N  represents  the  set  of  natural  numbers.  R  and 
]R>(j  denote  the  fields  of  real  numbers  and  nonnegative  re¬ 
al  numbers,  respectively.  R"  is  the  vector  space  of  all  71- 
tuples  of  real  numbers,  R"xr!  is  the  space  of  n  x  n  ma¬ 
trices  with  real  entries.  S"xn  is  the  set  of  real  symmet¬ 
ric  positive  definite  n  x  n  matrices.  The  notation  P  >-  0 
(P  -<  0)  means  P  is  real  symmetric  and  positive  definite 
(negative  definite).  AT  denotes  the  transpose  of  A,  and  we 
let  Sym(A)  =  AT  +  A.  In  symmetric  block  matrices,  we  use 
*  as  an  ellipsis  for  the  terms  that  are  introduced  by  symmetry. 
diag{- •  • }  denotes  a  block-diagonal  matrix.  ||-||  stands  for 
the  Euclidean  norm.  The  bounding  ellipsoid  is  expressed  by 
£{R)  =  {t  £  I"  xrRx  <  1  ,R  €  S"xn},  and  ball 
B(xo,6)  =  {x  £  R™  |  \\x  —  xo||  <  5,xo  £  R",<5  >  0}. 
The  right  derivative  of  a  matrix  function  F(x)  is  defined  by 
F(x)  =  lim/l_>,Q+  FU+hj~FU'>  _  por  ,-pg  sape  0p  sjmpiicity,  we 


denote  Af(A,  P,  P,  R,  a)  = 


ATP  +  PA  +  aP  * 
BtP  -aR 


II.  Switched  Systems  and  Output  Reachable  Set 

In  this  paper,  we  consider  a  continuous-time  switched  linear 
system  in  the  form  of 

£  :  x(t)  =  Aa{t)x(t)  +  Ba{t)u(t)  (1) 

2 /(f)  =  Ca(t)x{t)  (2) 


S  =  {ffcj-fceN,  where  to  is  the  initial  time  and  tk  is  the  fcth 
switching  instant.  Then,  we  define  Zt  =  {t  £  R>0  |  cr (f)  = 
i,i  £  A4}  to  denote  the  activation  time  interval  for  vth  mode. 
Obviously,  we  can  see  that  UieM  Tf  =  R>o  and  2}  0  Xj  =  0, 
for  i  ^  j,  Vi,j  £  M. 

The  output  reachable  set  of  system  (l)-(2)  is  defined  as 

Ry  =  { y(t )  £  Rna  |  x(t),y(t),x0,u(t)  satisfy 

(1))  (2)>  (3),  (4),t  £  R>0}  (5) 

The  following  lemma  introduces  the  main  idea  to  determine 
the  over-approximate  set  lZy  for  switched  system  (l)-(2). 

Lemma  1:  Consider  system  ( 1 )— (2)  under  initial  state  con¬ 
dition  (3)  and  input  condition  (4).  If  there  exist  a  family  of 
Lyapunov  functions  V*  :  Rn”  —X  R>o,  i  £  -M,  satisfying 
Vfitf)  =  0  and  Vj(x)  >  0,  Vx  f  0,  Vi  £  M,  matrices 
Ri,y  £  §”x",  i  £  A4,  and  scalars  a>0,  0</3<l  such  that 

Fi(t)  <  0,  Vt  £  £  M  (6) 

Gi,j(tk )  <  0,  Vffe  £S,ifi=  j,  Vi,  j  e  M  (7) 

Vi(x 0)  <  Vi  £  M  (8) 

xT(t)CjRi,yCix(t)  <  Vi(x(t)),  Vt  €  Zj,Vi  £  M  (9) 

where  Ffit)  =  Vi(x{t))  +  aVfixifi))  —  auT (t)Ruu(t)  and 
Gjj(tfe)  =  Vi (a;(ffc))  -  (3Vj(x(tf))  +  (3—  1.  Then,  the  output 
reachable  set  lZy  satisfies  lZy  C  Ry  =  l)ieM£(Ri,v)- 

Proof:  See  the  Appendix.  ■ 

Remark  1:  Conditions  (6)  and  (7)  actually  characterize  an 
invariant  set  O  =  [JieM  where  O,  =  {x(t)  £  R”x  | 
Vi(x(t))  <  1},  i  £  M.  By  (6),  it  leads  to  Vi(x(t))  <  0, 
Vtc(f)  £  Cli  =  {x(t)  €  R”x  |  Vi(x(t))  >  1},  this  guarantees 
that  once  the  state  x(t)  enters  f it  remains  in  it  during 
the  activation  time  of  the  ith  subsystem.  However,  (6)  is  not 
enough  to  ensure  x(t)  staying  in  V.  forever,  in  presence  of 
abrupt  changes  from  Vi(x(f£))  to  Vj(x(t^)),  where  i  f  j 
at  switching  instant  tk  £  S.  Thus,  (7)  is  necessary  to 
define  the  invariant  O.  It  ensures  that  Vi(x(t^))  <  1  when 
Vi(x(tf))  <  1,  that  means  the  switching  actions  will  not 
cause  x(t)  escaping  from  Cl.  In  addition,  (8)  implies  that  the 
initial  state  To  £  Xo  C  fi»,  and  (9)  estimates  the  output 

reachable  set  based  on  the  invariant  set  fl. 


where  x(t)  £  R™x  are  the  state  of  the  system,  and  the  initial 
condition  xo  belongs  to  a  bounded  ellipsoid: 

To  e  x0  4  e(Ro)  (3) 

and  u(t)  £  R"u  is  the  input  vector  which  is  assumed  to  satisfy 
the  following  ellipsoidal  constraint: 

7 fit)  £U  =  £{Ru),  Vt  £  R>0  (4) 

and  y(t)  £  R"a  is  the  output.  Define  index  set  A"f  = 
{1,  2, . . . ,  TV},  where  N  is  the  number  of  modes  and,  cr  : 
R>o  —X  JA  denotes  the  switching  function,  which  is  assumed 
to  be  a  piecewise  constant  function  continuous  from  right  and 
only  non-Zeno  swtichings  (i.e.,  the  switch  at  most  a  finite 
number  of  times  in  any  finite  time  interval)  are  considered  in 
this  paper.  The  switching  instants  are  expressed  by  a  sequence 


III.  Output  Reachable  Set  Estimation 

Although  Lemma  1  provides  a  general  framework  to  handle 
the  output  reachable  set  estimation  problem,  it  is  impracti¬ 
cal  for  actual  use,  since  it  does  not  provide  any  available 
computational  techniques  for  the  construction  of  Lyapunov 
functions  Vi(x(t)),  i  £  AL  Moreover,  the  proposed  condition 
(7)  requires  us  to  check  the  values  of  Lyapunov  functions 
at  all  the  switching  instant  tk  £  S.  However,  the  switching 
instant  sequence  S  usually  cannot  be  specified  in  advance, 
and  it  is  impossible  to  check  Lemma  1  for  all  the  switching 
instants  in  the  case  of  k  —X  00.  In  the  following,  numerically 
tractable  methods  are  presented  to  solve  the  output  reachable 
set  estimation  problem  in  the  framework  of  Lemma  1. 
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A.  Common  Lyapunov  Function 

One  natural  idea  to  analyze  switched  system  (l)-(2)  is  to  use 
common  quadratic  Lyapunov  function  Vi(x(t))  =  V(x(t))  = 
xT  (t)Px(t),  i  £  Ai,  to  avoid  checking  (7)  for  every  tk  £  S. 

Theorem  1:  Consider  system  (l)-(2)  under  initial  state  con¬ 
dition  (3)  and  input  condition  (4).  If  there  exist  matrices 
P  e  s^-xn-,  Ri  y  e  S"xn,  i  £  Ai,  and  a  scalar  a  >  0 
such  that 

if  (Ai,  B u  P ,  Ru,  a)  -<  0,  Mi  £Ai  (10) 

Cj RpyCi  A  P  -5  R0,  \/i  £  Ai  (11) 

then,  the  output  reachable  set  TZy  C  TZy  =  UieAt  £(pi,y)- 
Proof:  Construct  a  Lyapunov  function  in  the  form 
of  V(x{t))  =  xT(t)Px(t),  P  £  S”xn.  Let  us  con¬ 
sider  F(f)  =  V(x(t))  +  aV{x{t))  —  auT  (t)Ruu(t),  and 
along  with  the  trajectory  of  system  (l)-(2),  we  have 
F(t)  =  xT  (i)AT(Ai,Bi,P,Ru,a)x(t),  where  xT0)  = 
[xT(t)  wT(f)],  and  from  (10),  it  yields  F(t)  <  0,  Vf  £  K>o, 
so  that  (6)  holds. 

Then,  since  the  common  Lyapunov  function  is  chosen,  (7) 
automatically  holds  with  /?  =  1.  By  (11),  P  -<  IT,  ensures 
V(xq)  <  Xq  RqXq,  and  Cj Rt,yCi  -<  P,  i  £  Ai ,  guarantees 
xT (t)Cj RityCix{t)  <  V(x(t)),  Vf  £  K>o,  Vi  £  Ai,  that 
is  (8)  and  (9)  hold.  Thus,  by  Lemma  1,  we  have  the  output 
reachable  set  7Zy  C  Ky  =  UieM£(RLy)-  ■ 

Remark  2:  The  set  TZy  is  usually  expected  to  be  as  small 
as  possible  to  achieve  a  precise  estimate  of  reachable  set  TZy. 
Based  on  Theorem  1,  one  may  add  an  additional  constraint 
that 

Ri,v  h  cl i  e  >  0,  Mi  £  M  (12) 

which  implies  that  eyT  {t)y(t)  <  yT  (t)Ri,yy(t)  <  1,  namely 
y(t)  £  U ieM£(Ri,y)  Q  13(0, 1/y/e),  Vf  £  R>0,  SO  we  have 
to  maximize  e  to  obtain  the  smallest  ball  ,8(0, 1/s/e)  by 

max  e  s.t.  (10),  (11)  and  (12)  (13) 

Moreover,  due  to  the  existence  of  the  tuning  parameter 
a ,  the  result  in  Theorem  1  and  corresponding  optimization 
problem  (13)  are  not  standard  LMI  problems,  they  are  bilinear 
matrix  inequality  (BMI)  problems  and  known  to  be  NP- 
hard.  Fortunately,  several  algorithms  are  available  to  solve 
BMI  problems  such  as  the  iterative  linear  matrix  inequality 
(ILMI)  approach  in  [25],  [26],  or  using  numerical  optimization 
algorithms,  such  as  fminsearch  [13]  or  genetic  algorithm 
(GA)  [18]  in  the  optimization  toolbox  of  Matlab. 


tk}  is  called  the  dwell  time  of  tr(f),  and  PTmin  =  {cr(f)  |  <r  : 
R>o  —>  Ai,  ffc+i  —  tk  >  t min ,  Mk  £  N}  denotes  the  set  of  all 
switching  policies  with  dwell  time  greater  than  Tm;n. 

We  consider  a  class  of  time-scheduled  multiple  Lyapunov 
functions  inspired  by  [28]— [3 1  ]  as  follows: 

Vi(x(t))  =  xT (t)Pi(t)x(t),  f  £  K>o,  i  £  Ai  (14) 

where  Pi(t)  £  S™xn,  i  £  Ai  have  the  following  structure: 

Consider  the  interval  [f fc ,  ffc  +  rmin),  we  partition  it  into 
L  segments  described  as  Cu^q  —  [ffc  +  0q,tk  +  9q+ 1),  q  = 
0, 1, . . . ,  L  —  1  of  equal  lengths  h  =  rmin/L,  and  then  9q  =  0 
and  6q  =  qh  =  qrmin/L.  We  consider  a  class  of  continuous 
matrix  function  Pj(f),  t  £  [ffc, ffc  +  Tm;n)  chosen  to  be  linear 
within  each  segment  Ck,q,  q  =  0, 1, . . . ,  L  —  1.  Explicitly,  we 
can  see  that  U„=o  Ck,n  =  [4, 4+ A min)  and  Ck,nCCk,m  =  0- 
n  m.  Letting  Pi  q  =  Pi(f±  +  9q),  then  since  the  matrix 
function  Pft)  is  piecewise  linear  in  [ffc, ffc  +  rmin),  it  can 
be  expressed  in  terms  of  the  values  at  dividing  points  using 
a  linear  interpolation  formula,  that  is,  for  0  <  p  <  1,  q  = 
0,1,...,L— 1, 

Pi(t)  =  Pi(fi)  =  (1  -  p)Pi,q  +  pPi,q+ 1,  f  e  Pk,q,  i  £  Ai 

(15) 

where  p  =  L(t  -  tk  -  0q)/Tm in. 

As  a  result,  the  continuous  matrix  function  P,(f)  £  S"xn, 
i  £  Ai  can  be  completely  determined  by  Plyi  £  S’lxn,  q  = 
0,1 ,L,  i  £  Ai,  in  interval  [ffc,  tk  +  rmin).  Then,  due  to 
[ffc,  ffc+Tmin)  c  [ffc,  ffc+1),  for  the  remaining  time  in  [ffc,  ffc+i) 
denoted  by  Ck,L  —  [ffc, min,  ffc+i),  P:(f),  i  £  Ai  is  set  to  be 

P»(f)  =  P%, Li  t  £  Ck,L,  i  £  Ai  (16) 


In  summary,  Pj(f),  i  £  Ai  is  defined  as 


|  Pi(p),  fe4,„  ?  =  o,i,...,f-i 

l  Pi  ,Li  t  £  Ck,L 


(17) 


where  /j  is  defined  in  (15). 

Theorem  2:  Given  a  dwell  time  rmin  >  0  and  consider 
switched  system  (l)-(2)  with  cr(f)  £  PTmin  under  initial  state 
condition  (3)  and  input  condition  (4).  If  there  exist  matrices 

pi  q  €  gn.xn^  q  =  o,  1,  ,  L  i  £  M,  Rhy  £  S”xn,  i  £  Ai, 

and  a  scalar  a  >  0  such  that  for  Mi,  j  £  Ai 


A£(Ai,  Bi,  Pi,q,  Ru,  a)  +  ^  i,q  A  0,  q  —  0,...,L—l  (18) 

AC  (Ai ,  Bi,  P%,q~ i-i ,  Rui  A)  A  m  i,q  A  0,  q  =  0, . . . ,  L  1 

(19) 


B.  Multiple  Lyapunov  Functions 

Switching  actions  are  able  to  significantly  affect  the  evolu¬ 
tion  of  switched  systems,  for  example  the  instability  arises  as 
a  result  of  a  rapid  switching  between  stable  subsystems.  Simi¬ 
larly,  the  switching  rate  has  a  great  impact  on  the  reachable  set 
as  well.  Thus,  given  a  switching  rate,  how  to  estimate  the  set 
TZy  is  one  of  the  basic  problems  for  reachable  set  estimation. 
In  this  work,  the  concept  of  minimum  dwell  time  is  given  to 
constrain  the  switching  rate. 

Definition  1:  [27]  Given  a  switching  signal  function  <j(t) 

with  a  generated  switching  sequence  S,  rm ;n  =  inffceN{ffc+i  — 


Af(Ai,Bi,Pi  L,Ru,a)  ~<  0  (20) 

Pi, o  -  Pj,l  A  0,  if=j  (21) 

Pi, o  —  f?o  A  0  (22) 

Cj RhyCi  -  Pi>q  A  0,  q  =  0, . . . ,  L  (23) 


where  =  diag{P(Pijg+i  -  P:.9)/rmi„,  0}.  Then,  the 

output  reachable  set  TZy  C  TZy  =  (j,eM  £(Ri,y)- 

Proof:  Construct  a  Lyapunov  function  as  V  (f)  = 

J2ieM  ^i(t)xT (t)Pi(t)x(t),  where  P;(f),  i  £  Ai,  is  defined 
by  (17)  and  &  :  R>0  ->■  {0,1}  and  =  1  is  the 

indicator  function  representing  the  active  modes  at  time  f. 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


355 


IEEE  TRANSACTIONS  ON  AUTOMATIC  CONTROL,  VOL.  XX,  NO.  XX,  XX  XXXX 


4 


TABLE  I 

Computational  Complexities  of  Theorem  2  with  a  Fixed  a 


Number  of  Decision  Variables 

LMI  Constraints  Size 

nN(L  +  l)(n  +  l)/2 

n{N'2  +  2N  +  3  L) 

First,  let  us  consider  Fi(t)  =  V(t)+aV(t)—auT  (t)Ruu(t), 

which  can  be  rewritten  to 

Fi(t)=xT(t)(J?(Ai,Bi,Pi(t),Ru,a)  +  *i(t))X(t )  (24) 

where  xT (f)  =  [xT(0  wT (t)]  and  \l/j(f)  =  diag{Pj(i),  0}. 
Suppose  a(t)  =  i,  t  €  C-k,q,  Q  =  0, . . . ,  L  —  1,  one  has 

(-A*)  Bj,,  Pi(t),  Ru,  a)  =  (1  —  2  (25) 

where  Su  =  «£? (At,  Bu  Pitq,  Ru,  a)  and  Sij2  = 

J£(Ai,  Bi,  PitQ+i,  Ru,a).  Furthermore,  we  can  see  that 
Pi(t)  =  {Pi,q+ 1  -  Pi,q )(i,  t  G  Ck,q,  q  =  0, . . . ,  L  -  1,  and 
because  of  p  =  L(f— ffc— #g)/Tmin,  it  implies  that  /i  =  L/rmin, 
leading  to  Pj(f)  =  t  G  <7  =  0, . . .  ,L  —  1.  Thus, 

by  (18),  (19),  it  leads  to 

Fi(t)  <0,  Vf  G  M  Ck,n  =  [tk,tk  +  Tmin)  (26) 

Then,  we  consider  f  G  Ck,L-  Since  Pt(t)  =  Plti,  t  G  £k,L, 
we  have  Pi(t)  =  0,  Vf  G  Ck,L,  thus  (20)  guarantees  that 

Fi(t)  <  0,  Vf  G  Ck,L  (27) 

Thus,  from  (26)  and  (27),  we  can  conclude  that  Ft(t)  < 
0,  Vf  G  2 j,  Vi  G  AT  which  means  (6)  in  Lemma  1  holds. 
Next,  (21)  ensures  (7)  holds  with  3  =  1  and  (22)  guarantees 
(8)  holds  Finally,  we  consider 

GjR^yCi-Piit) 

=(1  -  p){Cj Ri,yC.i  -  Pi>q)  +  ^{Cj Ri<yCi  -  Pi,q+ 1) 

and  (23)  ensures  that  Cj R.,,yCi  —  Pi (t.)  <  0,  Vf  G  K>o,  Vi  G 

M,  which  implies  (9)  holds.  Therefore,  we  have  the  output 
reachable  set  lZy  C  Fy  ~  UieM  £{Ri'V)  by  Lemma  1.  ■ 

Remark  3:  Some  remarks  on  parameter  L  are  given. 

(1)  Parameter  L  implies  the  number  of  segments  consisting 
of  the  dwell  time  interval  [ffc,ffe  +  rmin).  A  larger  L 
yields  a  finer  division  of  [ffe,ffc  +  Tmin),  and  a  less 
conservative  result  can  be  consequently  obtained,  which 
will  be  demonstrated  by  a  numerical  example  later. 
However,  the  computational  cost  increases  as  L  grows, 
since  a  larger  L  inevitably  introduces  more  decision 
variables  and  LMI  constraints,  see  Table  I  for  the 
computational  complexity  analysis  for  Theorem  2  for  an 
H-dimensional  switched  system  consisting  of  N  modes. 

(2)  Similar  as  the  methods  adopted  in  [20],  a  piecewise  ma¬ 
trix  function  P,  (/r)  in  (15)  with  a  sufficiently  large  L  is 
able  to  approximate  a  generic  continuously  differentiable 
Pj(f)  with  adequate  accuracy  over  the  finite-time  interval 
[tk,tk  +  Tmin).  In  other  words,  if  L  — >  oo,  conditions 
(18)— (23)  in  Theorem  2  can  be  expressed  as  follows  with 


i,j  GM  and  f  G  [0,rmi„) 


Pi{t )  A  0 

(28) 

Af(Ai,  Bi ,  Pi(t ),  Ru ,  a)  +  Ti(f)  -<  0 

(29) 

T£ (Ai ,Bi,Pi (r min) ,  Ru ,  a)  ~<  0 

(30) 

Pi{ 0)  ^  Pj  (Tmin )  A  0,  1  ±  j 

(31) 

Pi(0)  —  Rq  -<  0 

(32) 

Cj Ri,yCi  -  Pi(t)  A  0 

(33) 

where  T^f)  =  diag{P;(f),  0}.  It  should  be  noted  that 
the  above  differential  linear  matrix  inequality  (DLMI) 
(28)-(33)  can  achieve  the  result  with  least  conservative¬ 
ness  in  our  framework,  but  it  is  not  numerically  tractable 
due  to  the  presence  of  continuous  matrix  functions  P,  (f). 

(3)  In  another  extreme  case  with  L  =  0,  P,.q,  shrinks  to 
Pi,  moreover,  due  to  (21),  we  have  to  choose  P,  =  P7, 
i  j.  Thus,  Theorem  2  is  reduced  to  Theorem  1,  namely 
the  common  Lyapunov  function  result. 

Given  an  L,  the  smallest  ball  B( 0,  l/\/e)  containing  the  tra¬ 
jectories  of  output  y(t)  in  the  framework  of  our  approach  can 
be  obtained.  Based  on  Theorem  2,  an  optimization  problem 
can  be  formulated  by  adding  (12)  with  ( 1 8)— (23)  as  follows: 

max  e  s.t.  (12)  and  (18)  —  (23)  (34) 

C.  Example 

Consider  a  switched  system  with  two  subsystems  as 


'  -2  1 

'  -1  0 

A  i  ' 

0  -0.9 

^2 

-1  -1 

= 

3  1 

7 

"bT 

= 

2  3 

1  0 

c2 

1  0 

0  1 

0  1 

The  initial  state  is  assumed  to  satisfy  Xq  G  {xo  G  R2  | 
||x0||  <  1}  and  the  input  is  assumed  to  satisfy  u(t)  G  {u(t)  G 
R  |  — 1  <  u(t)  <  1,  Vf  G  R>o},  which  implies  that  Rq  = 
diag{l,  1}  and  Ru  =  1. 

First,  we  use  Theorem  1  to  estimate  the  reachable  set 
TZy  contained  in  the  ball  B{ 0,  6)  with  the  minimal  6,  where 
S  =  1 ! yfe.  The  minimal  S  is  2.9033  obtained  by  solving 
optimization  (13)  with  the  aid  of  fminsearch.  It  should  be 
noted  that  this  result  is  applicable  for  the  arbitrary  switching, 
since  the  common  Lyapunov  function  approach  is  employed. 

Next,  if  the  dwell-time  constraint  is  further  considered  in 
the  switching  signal,  we  can  apply  Theorem  2.  Suppose  dwell 
time  rmin  =  1,  we  solve  optimization  problem  (34)  to  obtain 
the  minimal  S  with  L  =  1,  2, . . . ,  10,  which  are  depicted  in 
Fig.  1.  The  following  two  points  can  be  observed  in  Fig.  1, 
which  are  consistent  with  Remark  3. 

1)  The  value  of  6  monotonically  decreases  as  L  increases. 
This  means  that  a  less  conservative  result,  namely  a 
smaller  <5,  can  be  obtained,  if  a  greater  L  is  chosen. 

2)  The  L  =  0  is  equivalent  to  the  result  of  common  Lya¬ 
punov  function  approach,  but  it  is  more  restrictive  than 
the  result  obtained  by  the  multiple  Lyapunov  function 
approach  with  L  >  1. 
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□  ■  By  Multiple  Lyapunov  Function 
■  By  Common  Lyapunov  Function 

2.88  - .  .  .  .  . . . . . . . - 

2.86  - .  . □ .  .  .  . . . . . . . - 

’w  2.84  - .  .  .  .  . . . . . . . - 

2.82  D 
28 

D  □ 

2.78  - .  •  '  O- g p  . .  . . - 

□ . □ 

2.76 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 1 - 

01  23456789  10 

L 

Fig.  1.  Minimized  5  =  l/\/e  by  the  common  Lyapunov  function  approach 
(Theorem  1)  and  the  multiple  Lyapunov  function  approach  (Theorem  2)  with 
respect  to  L  =  0, 1,  2, . . . ,  10. 


-5  -4  -3  -2  -1  0  1  2  3  4  5 

y. 

Fig.  2.  1000  randomly  generated  state  trajectories  are  bounded  in  the 

estimated  output  reachable  set  IZy  =  7^i?2/  U 


Finally,  the  bounding  ellipsoids  'R,\  .y  and  ’TL-i.y  obtained 
with  L  =  10  are  shown  in  Fig.  2.  The  switching  signal  has 
tk+i  —  tk  =  1  +  rand,  k  £  N,  where  rand  is  a  random  number 
within  [0, 1],  thus  the  switching  signal  a(t)  £  _DTmin  with 
Tmin  =  1.  With  an  input  u(t)  =  sin (f),  1000  state  trajectories 
generated  from  1000  random  initial  states  from  a  unit  circle  are 
illustrated  in  Fig.  2.  As  Fig.  2  shows,  all  the  state  trajectories 
are  bounded  in  the  estimated  reachable  set  lZy  =  lZitVUlZ2,y, 
showing  the  effectiveness  of  our  approach. 

IV.  Safety  Verification  for  Uncertain  Switched 
System 

For  the  sake  of  being  concise,  we  focus  on  the  application 
of  Theorem  2  in  the  rest  of  this  paper,  since  Theorem  1  is  just 
a  special  case  of  Theorem  2  with  parameter  L  =  0,  see  point 
(3)  in  Remark  3. 


where  x(t)  £  Mnx  is  the  state  of  the  bisimilar  system,  the 
initial  state  xq  is  assumed  to  be  in 

x0  £  Xo  =  £(Ro)  (37) 

and  y(t)  £  ln»  is  the  output  of  the  bisimilar  system.  In  the 
rest  of  the  work,  the  input  u(t)  and  switching  signal  cr(f)  of 
E  is  considered  to  be  same  as  those  for  system  E. 

Definition  2:  [22]  A  relation  figs  C  x  MWx  is  called 

a  b -approximate  bisimulation  relation  between  systems  E  and 
E,  of  precision  S,  if  for  all  C x(t),x(t ))  <E 

1)  ||y(f)  -  y(i)||  <S,  Vf  £  R>o, 

2)  Vu(t)  £  U,  Vx(f)  satisfies  E,  3x(t)  satisfies  E  such  that 

(x(f),x(f))  £  figs,  Vf  £  R>o, 

3)  Vu(t)  £  U,  Vx(f)  satisfies  E,  3x(t)  satisfies  E  such  that 
(x(t),x(t))  £  figs,  Vf  £  Rutl¬ 
and  we  say  systems  E  and  E  are  approximately  bisimilar  with 
precision  S ,  denoted  by  E  ~s  E. 

Define  the  following  notations  x(f)  =  [xT(f)  XT(f)]T, 
y(t)  =  y(t )  -  y(t)  and 


'  At 

0 

Bt 

cj 

0 

A 

A 

-cj  \ 

and  let  0  <  7  <  1,  we  define  -Ro(t)  =  diagU-Ro,  (1  — 7)i?o}- 

Since  E  and  E  share  same  switching  signal  er(f)  and  input 
u{t),  an  augmented  system  E  can  be  derived  from  E  and  E 
as  below 

E  :  x(t)  =  Aa(t)x{t)  +  Ba(t)u{t)  (38) 

y{t)  =  Ca{t)x{t)  (39) 

with  initial  state  Xq  £  X0  =  £(R0(7))  and  input  it(f)  £  U  = 

£{Ru). 

Because  \\y(t)  —  y(f)||  <  <5,  Vf  £  R>o  holds  if  and  only 
if  y(t)  £  B(0,<5),  Vf  £  R>o,  the  problem  of  computing  the 
distance  S  between  E  and  E  can  be  converted  to  the  problem 
of  output  reachable  set  estimation  for  augmented  system  E. 

Theorem  3:  Given  a  dwell  time  rm jn  >  0  and  consider 
switched  systems  E  by  (l)-(2)  and  E  by  (35)-(36)  with  <r(f)  £ 
D Tm[n  under  initial  state  condition  (3),  (37)  and  input  condition 
(4).  If  there  exist  a  set  of  matrices  Pi  q  £  §d^+n*)x ("*+"■*) ^ 
q  =  0, 1, . . . ,  L,  i  £  M  and  scalars  a>0,  0  <  7  <  1,  e>0 
such  that  for  \H.  j  £  M. 

(Aj ,  Bi,  Pitq,  Ru,  or)  +  Hfi.g  A  0,  q  =  0, . . . ,  L  —  1  (40) 
Bi,  Pi  q^-i,  Ru,  ct)  +  A  0,  q  —  0, . . . ,  L  1 


(41) 

fifi(Ai,  Bi,PitL,Ru,a)  A  0  (42) 

P%,o  ~  Pj,L  A  0,  j  (43) 

Pi, 0  -  i?o(7)  A  0  (44) 

eCjCi-P^q  A0,  q  =  0,...,L  (45) 


A.  Approximate  Bisimulation 

For  a  continuous-time  switched  linear  system  E  de¬ 
scribed  by  (l)-(2),  an  approximately  bisimilar  continuous-time 
switched  linear  system  E  is  considered  in  the  following  form 


where  =  diag{L(P,;,(?+i  -  Pi,g)/Tmin,  0}.  Then,  we  have 
an  approximation  bisimulation  relation  .fig 5,  S  =  1  /^/e  such 
that  E  E. 

Proof:  Since  the  initial  states  x0  £  £(Rq)  and  xq  £ 
£{Ro),  the  initial  state  xo  satisfies 

x 0  Ro(j)x0  =  fixj R0x0  +  (1  -  7)5{[ Roxo  <1,  0  <  7  <  1 

(46) 


E  :  x(f)  =  AaWx(t)  +  BaWu(t)  (35) 

y{t)  =  Ca{t)x(t)  (36) 
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Thus,  it  means  that  xo  £  £(Ro(j)),  0  <  7  <  1. 

From  Theorem  2,  it  implies  that  the  output  reachable  set 
of  E  can  be  estimated  by  \JieM  £(e/)  =  0,  5),  where  5  = 

l/y/e,  so  the  output  y(t)  £  i3(0,  <5),  Vt  £  M>o-  Furthermore, 
due  to  y(t )  =  y(t)  —  y(t),  we  have  \\y(t)  —  y(f)||  <  5,  Vi  e 
K>o,  along  with  the  trajectories  x(t),  x(t)  generated  by  E 
and  E.  The  approximation  bisimulation  relation  such  that 
E  S  can  be  established.  ■ 

The  choice  of  a  larger  L  in  Theorem  3  will  lead  to  a 
less  conservative  analysis  result,  the  result  with  the  least 
conservativeness  can  be  deduced  by  letting  L  — >  00,  which 
is  however  numerically  intractable.  For  the  particular  case 
with  L  =  0,  Theorem  3  is  reduced  to  a  result  by  the 
common  Lyapunov  function  approach,  but  it  can  be  used  for 
the  arbitrary  switching  case. 

B.  Safety  Verification 


where  R0(y)  =  diagj7.R0,  (1  -  7)-Ro}.  and 


w(») 
1—1 i,q,l 


w(«) 

Vi,q  = 
-00  _ 


— Sym(17i(s))  +  aPi,q  +  ^>i,q  *  * 

-(Vj(s))T  - aRu  * 

Pi,q  +  Qi  -  (U^)T  -Vp  Sym(Qi)  _ 

— Sym(£/P)  +  aPi  ,9+1  +  4 'i,q  *  * 

-(ps))T  -aRu  * 

Pi,q+i  +  Qi  -  (Uls))r  -L/s)  Sym(Qi) 
L(Pi}q-\- 1  P%,q)  /  Tmin 

— Sym(t/i(s))  +  aPi,L  *  * 

-0p°)T  - aRu  * 

Pi,L  +Qi-  (t/P)T  -V}a)  Sym(Qj) 


XiA\s)  Mi  1  (,)  =  r  XiB\s)  +  Ni 

ZiA^  Mi  \  ’  i  |_  ZiB\s)  +  Ni 


wV  =  [  c\a)  -Sj  ]  ,  Qi 


Xi  Yi  ' 

Zi  Yi  _ 


We  consider  the  system  matrices  of  switched  system  E  are 
uncertain  and  satisfy  that  [A>  Bi  Cj]  £  93,;,  where 

=  co  {[A«  B™  (C,,-1))T], . . . ,  [A,(S)  B\S] 1  (cf  })T]} 

(47) 

where  co{  }  is  the  convex-hull  operator. 

Definition  3:  Consider  system  E  described  by  (l)-(2)  and 
(47)  with  Cp  =  I,  Vs  =  1, . . . ,  S,  Vi  £  Ad.  System  E  is  said 
to  be  safe  with  respect  to  the  unsafe  region  f 1U,  if  TZyC\Llu  =  0. 

Let  E  be  an  approximately  bisimilar  system  such  that 
E  ~,5  E.  Denote  TZy,  1Zy  the  output  reachable  sets  of  E 
and  E  respectively,  then  it  can  be  seen  that  TZy  CA r(Ky,6), 
where  Af{-,5)  denotes  the  5-neighborhood  of  a  set.  Conse¬ 
quently,  to  prove  that  E  is  safe,  it  is  sufficient  to  verify  that 

nyoN{Du,8)  =  %. 

Proposition  1:  If  E  E,  then  IZy  fl  A f(Slu,6)  =  0  => 
7 Zy  fl  Clu  =  0.  Namely,  E  is  safe  with  respect  to  A 5)  => 
E  is  safe  with  respect  to  f lu. 

In  the  following,  a  theorem  is  presented  to  compute  the 
system  matrices  for  a  bisimilar  system  for  uncertain  switched 
system  E. 

Theorem  4:  Given  a  dwell  time  rm;n  >  0  and  consider 
uncertain  switched  systems  E  by  (l)-(2),  (47)  and  E  by 
(35)-(36)  with  <r(f)  £  27rmin  under  initial  state  condition 
(3),  (37)  and  input  condition  (4).  If  there  exist  a  set  of 
matrices  e  R"xXnx,  £  ]Rn*xri«)  Xt  £  I%xn*, 
Yi  £  R%x%,  Zi  £  RnxXnx,  Si  £  M"**"*,  pi  q  £  g^»xX2nX) 
q  =  0, 1, . . . ,  L,  i  £  M.  and  scalars  a  >  0,  0  <  7  <  1,  5  >  0 
such  that  for  Vi,  j  £  M.  and  Vs  =  1, 2, . . . ,  S, 


SpP  -<  0,  q  =  0, ...  ,L  —  1 
q  =  0, . . .  i  L  —  1 

"(s)  -<:  0 

A  u 

Pi, 0  -  Pj,L  A  0, 

Pi, 0  -  Roh)  A  0 


r  —  p 

J  i,q 


—S2I 


-<  0,  q  =  0,...,L 


(48) 

(49) 

(50) 

(51) 

(52) 

(53) 


Then,  we  can  obtain  an  approximately  bisimilar  system  E 
in  the  form  of  (35)-(36)  and  an  approximation  bisimulation 
relation  such  that  E  E,  where  the  corresponding  system 
matrices  are 

[  Ai  :  Bi  :  CJ  }  =  [  Y~1Mi  ;  Y-^N,  ;  Sj  }  (54) 

Proof:  First,  Qi  +  Qj  -<  0  implies  Y,t  +  Y.A  ~<  0,  thus 
Yi  is  nonsingular.  Then,  substituting  Mt  =  AjYt,  Ni  =  Bt  Yt 
and  Si  =  Ci  into  (48),  it  becomes 

-Sym(QiAP)  +  ctPi,q  +  ^i,q  *  * 

—  (Bls))TQj  -aRu  *  A0 

Pi,q  +  Qi-{MS))TQJ  ~QiB\s)  Qi+Qj  _ 

By  left-multiplying  the  third  row  of  above  inequality  by 
(Ap)T  or  (Ap)1  and  adding  it  to  the  first  or  second  row, 

and  right-multiplying  the  third  column  by  Ap  or  Ap  and 
adding  it  to  the  first  or  second  column,  it  yields 

Sym(Pji9AP)  +  aPitq  +  '&i,q  *  * 

(B(iS))TPi,q  -aRu  *  A  0 

Pi,q  +  Ql  -  QiA\s)  -QiB\s)  Qi  +  Qi  _ 

Due  to  (47)  and  simple  convexity  arguments,  the  above 
inequality  ensures  (40)  holds.  Through  a  similar  proof,  it  can 
be  found  that  (49)  =>  (41)  and  (50)  =>  (42).  Moreover,  (51) 
and  (52)  are  equivalent  to  (43)  and  (44). 

Finally,  letting  e  =  1/52  and  by  Schur  complement,  (53) 
ensures  that  (45)  holds.  Therefore,  the  approximation  bisimu¬ 
lation  E  E  can  be  established  by  Theorem  3.  ■ 

Given  an  L,  the  optimized  approximately  bisimilar  system 
Eopt  can  be  obtained  by  minimizing  the  precision  S  by 

min  52  s.t.  (48)  -  (53)  (55) 

So  far,  according  to  Proposition  1,  we  can  perform  the  safety 
verification  for  uncertain  system  E  with  respect  to  ( via 
verifying  the  safety  specification  of  the  bisimilar  system  E 
with  respect  to  the  set  Af(Clu,S),  the  5-neighborhood  of  £lu. 
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TABLE  II 


Precision  5  with  L  =  1,2, 3, 4, 5  and  Computation  Time  (C.T.) 
with  a  Fixed  a 


L  =  1 

L  =  2 

L  =  3 

L  =  4 

L  =  5 

s 

0.459 

0.434 

0.425 

0.414 

0.409 

C.  T. 

0.573  s 

0.862  s 

1.221  s 

4.762  s 

15.263  s 

C.  Example 

In  this  subsection,  the  safety  verification  for  an  uncertain 
switched  affine  system  x(t)  =  Ai(t)x  +  bi,  i  £  {1,2},  is 
considered.  The  system  matrices  are  given  as  blow: 


r  a  1  1 

-2  1 

r  A-,  1 

’  -1  7 (*)  ’ 

XT 

= 

1 

O 

co 

? 

XT 

= 

-1  -1 

L  °i 

3  1 

L  °2  J 

2  3 

where  7 (t)  £  [0,0.1]  is  an  uncertain  time-varying  parameter. 
The  initial  state  is  assumed  to  be  xq  £  {20  £  R2  |  ||xo||  < 
0.1},  which  implies  that  R0  =  diag{100, 100}.  The  switching 
signal  is  a  periodic  switching  law  as  tk+i  —  tk  =  1,  Vfc  £  N. 

Using  Theorem  4,  a  switched  system  with  exact  parameters 
can  be  obtained,  with  a  corresponding  precision  S.  One  point 
needs  to  be  clarified  here  is  that  (50)  can  be  removed  for 
this  particular  periodic  switching  case,  since  (50)  exactly 
corresponds  to  the  interval  [t*.  +  rmjn)oo)  which  does  not 
appear  at  all.  Similar  to  the  experimental  results  for  reachable 
set  estimation  (Section  III,  C),  the  precision  <5  tends  to  a 
smaller  value  as  a  larger  L  is  chosen  to  apply  Theorem  4, 
see  Table  II  for  L  =  1,2, 3, 4, 5. 

Then,  in  order  to  validate  our  approach,  we  first  let  L  =  1 
and  obtain  the  corresponding  system  matrices  as  follows: 


Fig.  3.  The  safety  verification  via  SpaceEX  for  the  certain  system  derived 
with  (L  =  1).  The  blue  area  is  the  reach  set  computed  by  SpaceEX,  and  the 
yellow  lines  are  the  random  state  trajectories.  The  safe  or  unsafe  property  of 
the  original  uncertain  system  cannot  be  concluded  since  the  reach  set  of  the 
certain  system  intersects  with  the  new  unsafe  region. 
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With  the  above  switched  system  with  exact  parameters, 
we  can  conduct  the  verification  for  the  uncertain  switched 
system.  Given  three  unsafe  regions  =  B{[ 0.7  1.7],  0.6), 
nUt2  =  B([ 2  -0.2], 0.5)  and  f 2U,3’=  £([3.5  1.5], 0.9), 
the  new  unsafe  regions  are  described  by  their  neighborhoods 

nUil  =  AT(Slu,i,  0.459),  =  -V(n„,2,0.459)  and  Uu,3  4 

■V(^u,3, 0.459).  Thus,  the  verification  for  uncertain  switched 
system  can  be  done  via  verifying  if  the  new  system  is  safe 
with  respect  to  the  new  unsafe  regions.  We  can  use  SpaceEx 
[32]  to  perform  the  verification  for  the  certain  system. 

The  verification  result  is  illustrated  in  Fig.  3.  However,  the 
safety  of  the  original  system  cannot  be  guaranteed  since  the 
computed  reach  set  intersects  with  U„  3.  Then,  we  let  L  =  5 
which  produces  a  smaller  precision  6,  and  the  system  matrices 
are 
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In  comparison  with  Fig.  3,  this  smaller  5  yields  small¬ 
er  unsafe  regions  as  f2u  3  =  Af(Qui,  0.409),  TlUy 2  — 
AT(UUi2,  0.409)  and  3  =  A/"(flu  3, 0.409).  By  the  results  in 
Fig.  4,  we  can  conclude  the  safety  of  the  uncertain  switched 
system. 


Fig.  4.  The  safety  verification  via  SpaceEX  for  the  certain  system  derived 
with  (L  =  5).  The  safety  of  the  original  uncertain  system  can  be  concluded 
since  the  reach  set  of  certain  system  has  no  intersection  with  the  new  unsafe 
regions. 


V.  Conclusions 

In  this  paper,  the  output  reachable  set  estimation  problem 
for  switched  linear  systems  has  been  investigated.  With  the 
aid  of  the  common  Lyapunov  function  and  multiple  Lyapunov 
function  approaches,  the  output  reachable  set  can  be  over¬ 
approximated  by  a  set  of  bounding  ellipsoids.  Moreover,  a 
sufficient  condition  for  the  existence  of  an  approximate  bisim¬ 
ulation  of  two  switched  linear  systems  is  proposed,  which  can 
be  viewed  as  an  output  reachable  set  estimation  for  the  system 
combining  the  two  bisimilar  systems.  Finally,  by  the  result  of 
approximate  bisimulation,  the  safety  verification  problem  for 
uncertain  switched  systems  can  be  dealt  with  by  verifying  the 
safety  of  its  bisimilar  system  with  exact  parameters.  In  this 
paper.  A,  are  required  to  be  Hurwitz  stable.  By  the  techniques 
used  in  [33],  the  result  in  this  paper  can  be  readily  extended 
to  the  case  with  some  Ai  are  unstable.  In  addition,  according 
to  Table  I,  the  computational  cost  significantly  increases  as 
the  system  order  and  number  of  modes  grows,  how  to  reduce 
the  computational  complexity  and  make  it  applicable  for  high 
dimensional  systems  with  large  amounts  of  subsystems  will 
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be  our  future  study. 
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Appendix 

Proof  of  Lemma  1:  Define  the  following  Lyapunov  function 
as  V{t)  =  where  &(t),  i  £  M.  is  same 

as  in  Theorem  2.  First,  we  consider  any  t  £  [tk,tk+ 1)  C  2 
Vi  €  M..  (6)  implies 

V(t)  <  -aV(t)  +  auT (t)Ruu(t),  t  £  [tk,tk+1)  (56) 

Then,  multiplying  both  sides  of  (56)  with  e“V_tC  and  then 
integrating  it  over  we  have  V(t)  <  e~a^t~tkW(t^)  + 

fk  e~a^~s^uT (s)Ruu(s)ds.  Due  to  u(t)  £  S(RU),  Vi  £ 
M>o,  that  is  uT (t)Ruu(t)  <  1,  Vi  £  R>o,  we  have  the 
following  result 

V(t)  <  e~a(t-tk)V(t+)  +  J  e~a{t-s)ds 

=  e~a{t-tk)V(t+)  +  1  -  e““(t-tfe) 

and  it  can  be  rewritten  to 

V(t)  -  1  <  e-a{t~tk\V(t+)  -  1),  t  £  [ifc , ife+i )  (58) 

Next,  we  consider  tk  £  S.  From  (7),  we  can  obtain  that 
VfJ)  <  /3V(tf)  +  1  —  /3,  tk  £  S ,  which  can  be  equivalently 
rewritten  to 

V(fi)  -  1  <  p(V(tf )  -  1),  tk  £  S  (59) 

Combining  (58)  and  (59),  the  following  derivation  can  be 
obtained  for  Vi  £  K>o 

V(t)  -  1  <  e-a(t-tfe)(V(i+)  -  1)  <  /3e~a(t-tk)(V(tf )  -  1) 

<  •  •  •  <  ^Numft-tojg-afi-to )(y(£Q)  _  1) 

where  Num(i  —  to)  denotes  the  number  of  switchings  during 
[io ,  i)  -  Due  to  a  >  0  and  0  <  /?  <  1,  it  means  that 

V(t)  -  1  <  V(t„)  -  1,  Vi  e  R>0  (60) 

Furthermore,  (8)  implies  that  V(io)  <  Xq  RqXo  <  1,  and  (9) 
together  with  (60)  yield  that  yT  (t)RitVy(t)  <V(t)<  1  holds 
when  <r(f)  =  i  £  AL  f  £  K>o-  For  all  possible  i  £  M.,  y(t) 
thus  satisfies  y[t)  £  \JieM  £(Ri,y),  Vi  £  K>o  and  therefore, 
TZy  C  TZy  by  the  definition  of  7Zy  given  in  (1). 
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8.  List  of  Symbols,  Abbreviations,  and  Acronyms 

•  CPS:  cyber-physical  system(s) 

•  DCPS:  distributed  cyber-physical  system(s) 

•  Hynger:  hybrid  invariant  generator 

•  HyperSTL:  hyperproperties  for  signal  temporal  logic 

•  HyST:  hybrid  source  transformation  and  translation  software  tool 

•  SLSF:  Simulink/Stateflow 

•  StarL:  stabilizing  distributed  robotics  language 

•  STL:  signal  temporal  logic 

•  RTA:  runtime  assurance 

•  UAS:  unmanned  autonomous  system 

•  UAV:  unmanned  aerial  vehicle 
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